Best practice for Windows data drive storage location

[Kevin8520]

Hello all! Long time listener, first time caller. I have a current gen HP server with 8 SSD drives that doesn't have PVE or Windows installed yet. I am planning on the following storage configuration:

2x mirrored drives as the boot drive
4x RAID10 drives as VM storage
2x mirrored drives to use as a backup drive for the VMs and Snapshots

I will be running two VMs - one Windows Server and one Windows 11. I need an "OS Drive" and a "Data Drive" on each VM.

What is the best practice to set this up? Is it better to create a disk size of 1TB in PVE and then partition that into a 250GB OS and a 750GB DATA during Windows installation, or is it better to create a disk size of 250GB in PVE for the OS and then "add hard disk" a 750GB drive from my ZFS RAID10 store?

I appreciate any input!

 

[tcabernoch]

ZFS is not for everyone. I understand if you don't even want to consider it. It took me a couple years.

But you did ask.
I'm suggesting a 2 vdev zpool. That's a bit of speed, a bit of redundancy.
It's nearly certain that somebody will promote some different options, so take this as a starting point.

Run the installer. When you hit the Disk screen, change it to raidz1. (Or go with raidz2 for extra redundancy.)​

Give it (only) the first 4 disks. Finish the install.​

​

zpool status is going to show you a 4 disk pool with a single raidz1 vdev.​

​

Use sgdisk to partition the remaining disks as covered here.​

https://pve.proxmox.com/wiki/ZFS_on_Linux#_zfs_administration​

​

Use this to get the disk IDs: ls -alh /dev/disk/by-id​

If you did the sgdisk part correctly, you'll be using the third partition on these disks.​

​

Add them as another vdev (of the same redundancy as the original, either raidz1 or raidz2.)​

zpool add rpool raidz1 <device1-part3> <device2-part3><device3-part3><device4-part3>​

What you have in the end is something that reads like 4 times the speed of a single disk and writes twice as fast and tolerates the loss of 1 disk in either vdev. (I wrote this off the cuff. Maybe somebody will chime in if I b0rked any of it.)

 

[_gabriel]

zfs Raid10 aka striped mirror is better for VM because better iops.
dedicated zfs pool for VM storage is more flexible.
Read :
https://forum.proxmox.com/threads/zfs-raid-10-mirror-and-stripe-or-the-opposite.101341/post-437271

 

[tcabernoch]

I agree with _gabriel completely, and would not follow their advice or recommend that you do so.
Do follow that link tho. I learned some things about advanced zfs tuning there. Gonna be chewing on that for a bit.

--------------------------
I believe that in practical terms, that suggestion meant to just select 2 disks for a raid 1 mirror during the install and then add 3 more vdevs later.
zpool add rpool mirror <device3-part3> <device4-part3>
zpool add rpool mirror <device5-part3> <device6-part3>
zpool add rpool mirror <device7-part3> <device8-part3>
The resultant zpool is 4 pairs of mirrors.
Yes. That's faster. Of course it is.
Each vdev acts like an old-skool stripe RAID member, increasing speed.
You've also wasted 4 disks.
And if ZFS looks like a headache now, wait till you get to replace a disk on the CLI. Your first attempt will have you biting your nails. This structure composed of mirrors is cumbersome, and it will have you tinkering with it again sooner than you'd like.

---------------------------

I think the reference to a dedicated VM pool means having a boot pool and later add dedicated storage with the remaining disks.
I believe this would create a second zpool. (Perhaps I'm mis-interpreting the original cryptic comment "dedicated zfs pool for VM storage is more flexible.")
I would recommend staying with a consistent zpool structure across all your hosts, so I don't think this sounds like a good idea.
Whatever you do with the physical geometry, you want your storage pool to look like "rpool/data/", on all hosts so you don't have to tinker with server-specific storage configs, which is actually a cluster-level setting. Live migration (vmotion) also depends on consistent storage pool structures.

 

[Kevin8520]

Here is my plan based off the feedback I have received. Let me know if I am way off base.

8 drives total - each drive is a 1.6TB SAS attached SSD.
1) During PVE installation I will select the first two drives and let PVE install as a ZFS mirror. I know this is a huge waste of space. I am unsure what I will do with the extra space at the moment.

2) Once PVE is up and running I will create a RAID10 ZFS using 4 drives and store all my VMs and logical drives that Windows will see on it.

3) I will use the last 2 drives to setup a ZFS mirror use them to store VM backups. I know the GUI won't natively allow this. I will use the following instructions https://aaronweiss.me/how-to-create-zfs-backups-in-proxmox/, which have worked in my lab environment.

Please let me know your thoughts. Thank you very much!

 

[tcabernoch]

That sounds like an informed decision.
I think from what you are saying, you are going to create 3 different zpools. It would look like this.

pool1
mirror-1

pool2
mirror-2
mirror-3

pool3
mirror-4

Great. That's a completely legit setup.