ZFS on storage only

[xvegax]

Hi guys,

I am absolutely new to Proxmox and also have never installed/configured a virtualisation server for hosting several VMs. So, I heard and thought, Proxmox seems a good idea for that, especially, as I like Debian a lot.

The system will be: Ryzen 7 3700X, 32 GB RAM, Gigabyte Mainboard, 1 TB NVMe Gen4, 2x 8 TB HDD.

The VMs will be four gaming servers (each VM one gaming instance on a single core), a database server, a file server, a voice server and a backup system.

Now, I have some questions about the installation:

1. Is it more sensible to use one VM for all four gaming instances and give it four cores or is my intial idea of using a VM for each better?

2. What is the best practise to install/format the drives in this setup with one NVMe and two HDDs? Would it be good to use the NVMe only for the system and the VMs and those two HDDs then just for storage, database, files, ... etc? Is there an easy tutorial for this scenario somewhere or any explaining help to find?

I must admit the file system question makes me a bit nervous, as I never have worked with ZFS, especially not on such a complicated environment (virtualisation server, I mean). I just hope to avoid initial errors and a waste of hours.

Thanks a lot in advance.

Best regards


PS The title should say "ZFS on storage only? (new installation)" with a question mark.

 

[LnxBil]

1. Is it more sensible to use one VM for all four gaming instances and give it four cores or is my intial idea of using a VM for each better?

4 Servers are better for isolation.

2. What is the best practise to install/format the drives in this setup

Normally, it's the other way around: You specify what you want in your setup and then buy hardware. A single disk without redundancy is never a good idea. Is your server running anything critical?

and those two HDDs then just for storage, database, files, ... etc?

database? If something has to go on an SSD, it's the database. Nothing will have more read/write cycles than your database. (if this is a normal database centric application of course.)

 

[xvegax]

4 Servers are better for isolation.

Normally, it's the other way around: You specify what you want in your setup and then buy hardware. A single disk without redundancy is never a good idea. Is your server running anything critical?

database? If something has to go on an SSD, it's the database. Nothing will have more read/write cycles than your database. (if this is a normal database centric application of course.)

Alright, I will do it with one VM for each gaming instance.

To be honest, there will be nothing critical on those VMs, at all, and also, the content is getting backupped daily/weekly to another server in a different location just to be sure. So, you would propose, rather to use two 500 GB NVMes than one 1 TB NVMe?

The database is not an active one, just some information, that is requested once per week or even biweekly. Otherwise I would have put it on a SSD with faster access, as you recommended.

Thank you very much for your help/effort.

But the installation/formatting is not clear to me yet. Is it like the NVMe/-s is getting Ext4 or anything else and the storage you provide with the ZFS format or would it be better to use ZFS for all drives, but put them in two pools or how does it work in such a scenario with one (or two) NVMe/-s and some HDDs?

 

[LnxBil]

So, you would propose, rather to use two 500 GB NVMes than one 1 TB NVMe?

No, I would propose two 1 TB NVMe SSDs, if you need 1 TB of space. The goal is redundancy here.

But the installation/formatting is not clear to me yet. Is it like the NVMe/-s is getting Ext4 or anything else and the storage you provide with the ZFS format or would it be better to use ZFS for all drives, but put them in two pools or how does it work in such a scenario with one (or two) NVMe/-s and some HDDs?

Everything depends on what you want. Please familiarize yourself with the possibilities: https://pve.proxmox.com/wiki/Storage

If you do not have a hardware raid controller and want redundancy, your only option is ZFS is a supported PVE environment.

IMHO, your hardware is kind of stange for me consisting of ultra fast SSD and spindles. There is nothing in between. Maybe a better option would have been having 4 or 6 SATA/SAS SSDs in a RAID5 or RAIDz1 configuration. Or even starting with 4 spindles without any SSD if the cost are too high. It's always a bad idea to buy hardware and then figure out what you can do with it, better the other way around.