[SOLVED] Best Practice PVE Fileserver

[cdn123]

Hello,
right now we have used pve with sw raid and ext4.
After reading a lot of forum posts and Reddit threads, most people suggest to use ZFS.
We have one new pve node for several Windows Fileserver VMs.

AMD Ryzen 7 3700X Matisse 8-core (Zen2) CPU 2 x 1 TB NVMe SSDs, 4 x 22 TB SATA HDDs, 64GB DDR4 ECC RAM

We configured SSDs as Raid1 and HDDs as Raid10.

What is your suggestion? Change to ZFS with mirrored special devices?
We would love to use the ssds for OS partitions of Windows VMs. But may it also make sense to use a part for metadata caching?

We haven‘t begun to migrate VMs so right now we can change the whole setup.

Thank you in advance!

 

[LnxBil]

Change to ZFS with mirrored special devices?

YES, but only if the SSDs are enterprise level. It'll boost your harddisks tremendously, yet 4 disks may just not enough for the workload you use, yet that depends - YMMV.

We would love to use the ssds for OS partitions of Windows VMs. But may it also make sense to use a part for metadata caching?

Luckily, you can do both. Read up on special_small_blocks, which does exactly what you want.

 

[cdn123]

YES, but only if the SSDs are enterprise level. It'll boost your harddisks tremendously, yet 4 disks may just not enough for the workload you use, yet that depends - YMMV.


Luckily, you can do both. Read up on special_small_blocks, which does exactly what you want.

Thank you. I guess I only would use the special device for metadata.
I also need to use the SSDs for pve. How do I proceed? Can I create an OS partition, vm partition and special device partition during installation of pve?

 

[LnxBil]

I guess I only would use the special device for metadata.

Depending on the actual data on the harddisks, you may need only a fraction of it. I can recommend to e.g. have your PVE root on it as well as anything that needs a little bit more power. Again, please look at special_small_blocks.

I also need to use the SSDs for pve. How do I proceed? Can I create an OS partition, vm partition and special device partition during installation of pve?

Install just on the harddisks, add the special device mirrored vdev, set special_small_blocks and afterwards send/receive your PVE ROOT once to transfer it to the SSDs (more precicesly the dataset with special_small_blocks)