New threadripper workstation build - NVME disk question / Performance related

[marklinton]

I'm in the process of building a new workstation build around the threadripper platform. I am going to be using 4x 2TB samsung 980 Pro NVME disks which are installed in a 4x nvme card (Asus).

I want to make sure that I'm getting the best performance from these drives as they will be used sporadically in a high disk io linux VM environment.

Based on my reading I should be setting these up in a striped ZFS pool with 4k block sizes. Any other tips for the drive installation and PVE configuration?

Thanks.

 

[Dunuin]

If you want to use ZFS its recommended to use enterprise grade SSDs and not consumer SSDs like the 980 Pros because they die too fast, can't handle sync writes well which are needed for the ZFS transactions and they got no powerloss protection so you could loose data on an power outage, power supply failure or kernal crash.

So better would be something like the Samsung PM1733, PM9A3, PM983 or PM963 if you want to stick to samsung. And the U.2 versions are preferable because they are faster and easier to cool than the M.2 version. There are M.2-to-U.2-cables so you can use U.2 drives connected to M.2 slots.

 

[marklinton]

Thanks @Dunuin, I already have the 980 Pros, so I'm wondering if it might be better just to do a traditional file system on them then, or maybe use the motherboard's RAID facility? I'm actually not too worried about data protection as I have a large spinning disk array that will hold a backup.

Cheers.

 

[Dunuin]

HW raid or backups will only help you if a complete drive dies but won't protect you from data corruption. Without a advanced filesystem like ZFS/btrfs/CEPH you basically will never know if the files are healthy or not so you might backup data that is already corrupted without knowing. Lets for example say bit rot hits you and entries in a database get corrupted. Maybe that DB stored some credit card numbers or something else and these numbers are now wrong because only some digits flipped position. If you dont bill those credit cards for a year you possibly won't recognize that these credit card numbers aren't valid anymore. So you basically backupped corruted data for a year without knowing. Know lets say you don't have space to store your backups for eternity so you overwrite old backups with new ones. Its very likely that if you recognize this corruptions after a year you don't have any backups that are older than a year so all your backups only contain these wrong creditcard numbers.
With ZFS this woukdn't be a problem. ZFS checksums everything while storing it and from time to time it will open all files, calculate the checksums again and compare them to the initial checksums. If these checksums don't match anymore ZFS knows that these files got corrupted by hardware failures or bit rot and it will warn you that corrutions happened and will tell you which files are effected. And if you also got some kind of parity (raid1/10/5/6) ZFS will heal itself and autorepair these corruted files based on the parity data.
So ZFS basically does the same for your SSD that ECC does for your RAM. You buy ECC RAM because you don't want that data corrupts while stored in RAM. So if you think you really need ECC RAM you also need ZFS.

Long story short... if you don't want that your data corrupts over time and you want to be sure that your stored data always keeps valid, these is no alternative to sruff like ZFS.