Mass storage - how to make it available to multiple vms?

[tymanthius]

Ok, so I just discovered proxmox in my quest to rebuild my home server. I like it.

One of things I haven't yet discovered in my reading of the last new hours is how to handle mass storage that multiple vm's will need access too.

This is all in one box, but the thought is to put Proxmox VE in, then run one VM w/ pfsense for firewalling/routing the whole house & VM's, one w/ Plex, another w Emby (gradual swap from Plex to Emby, eventually Plex will go away), another w/ some web interfaces for manageing plex/emby, and one w/ downloaders. Possibly more.

The hardware will be an i5-6600k, Z170 chipset, 8 or 16Gb of memory.

5x3TB hd's in RAID5, 1x240SSD, 1x160SSD

So, the Raid array is where my media will live, obviously.

What's the best way to share this data out to the various vm's that will need it?

I like the idea of running a NAS in VM that would do it, just so I could play w/ the software. But I want something that I can read the data directly from Proxmox cli too. That, and everything I've read so far says don't VM a NAS.


Any suggestions are welcome.

 

[pirateghost]

NFS for your VMS to attach to and use.

 

[tymanthius]

Makes sense.

Is there a How-To somewhere?

I'm not sure how to make it available . . . oh, wait, virt0 is really just a switch. So Proxmox IS on the network. I keep forgetting that. I keep thinking of it like the floor, and all the stuff is built on, but not really connected to it.

Still, a how to would make some nice reading.

 

[pirateghost]

Any Debian based how to would work

 

[tymanthius]

Okies. Just wanted to be sure there no extra bits for being on a VM. Thanks!

 

[pirateghost]

Don't make it a VM. You can use NFS directly on proxmox

 

[tymanthius]

I wasn't clear - for the clients being vm's on the host machine.

Easy peasy. I was over thinking it.

 

[Q-wulf]

There is multiple options via NFS:

• NFS on your proxmox host
• VM with any NAS-Software (trueNas, Freenas, Openmediavault, ...) and Passthrough of Physical Disks (Raid via Nas-Software)
• VM with any NAS-Software and passthrough of Hardware Raid-Controller (not sure will work on your hardware - check with "lspci" (terminal/console) for the controller-layout.
• VM with any NAS-Software residing on Virtual Disks (Virtio) on your Storage. Could be Raid5 via physical controller or Software Raid on the VM (this is what i'd do)

==> Attach em to Proxmox-Storage via Datacenter->Storage->Add-> NFS (data should be routed internally for better CPU/performance ratio you could use openvswitch)
Sidenote: difficulty easy-medium (depending on personal level)



ps.:
If you had at least 2 SSD's not being used for OS, i'd do a simple Single node Ceph-Cluster attach the 5 HDD's as HDD-OSDs. Create a HDD-Pool ontop of those. Either Replicated or EC-Coded (i'd do EC). Add the SSD's as SSD-OSDs and create a SSD-Pool ontop of it. Choose Replicated(1) type of pool. Add it as caching pool ontop of your HDD-pool.
Attach to Proxmox via krdb.
YOU'd get 12 TB Storage (with Parity=1 - 25% overhead) or TB Storage (with parity=2 - 66% overhead). At performance-levels of around half an SSD's regular Performance under heavy load.
Sidenote: medium-superhard (depending on personal level)
Big plus: If you need more space/performance/redundancy , you can Scale vertically (add more HDD/SSD) or horizontally (add more nodes - don't need to, but can be proxmox) no problem.

 

[tymanthius]

Your PS is almost gibberish to me. LOL

I get the basic idea - group the 5 storage drives, then put two SSD's on top as a cache. That gives about half the performance of an SSD, but all the space of the platters. Is that correct?

Honestly, I don't think I need that much performance, as the platter drives just hold my tv & movies, and the bottleneck there isn't likely to be the drives.

 

[Q-wulf]

[...]
I get the basic idea - group the 5 storage drives, then put two SSD's on top as a cache. That gives about half the performance of an SSD, but all the space of the platters. Is that correct?

Almost.
it basically gives you a tolerant system where you can loose (SSD-Number -1) in SSD's and (HDD's for Parity -1) in HDDs before you start loosing data. That is the selling Point. Its also done "natively", as you do not need to do passthrough of Raid-controllers/HDD's (which most NAS-Software providers advocate against) which is a big plus as there is less stuff that can cause issues). In Fact you do not need a Raid controller at all, since you'd attach all drives in JBOD-mode.
When you attach such a Pool to Proxmox via krdb, it is handled like a block storage device by Proxmox, so you can put your VM's Disks on it.

The added Bonus is:
that you can scale it without having to worry about wether your disks have the same size. What happens if you'd have to add a second Raidcontroller cause the first one ran out of Ports, or there is no more space in your Case (and you'd need to use a second Node)

The downside is:
- that it can be a rather steep learning curve.
- you cannot use the Proxmox build-in backup function into (k)rdb - you'd either need CephFS installed ontop of Ceph or a VM with a NFS-Server running ontop of it.
- you can do Snapshots of/in Ceph-Pools - but there is no integrated gui for it in Proxmox. Its all CLI

 

[tymanthius]

Well, I'm going to use software raid on the debian host, so that simplifies things. That's fine for my needs. Raid is serving as 'almost back up' as the media on it will never be important to me. It's just tv/movies in an easy to access manner. Time consuming to rebuild, but not a major loss. Raid 5 just saves me from losing that if a hd dies.

I think the learning curve is too steep today. I'm learning a LOT just to set this up.

But you have given me ideas for the future.