PROXMOX INSTALLED IN RAID1 (ZFS) TWO SSDs - VERY POOR PERFORMANCE

[VDNKH]

I have installed several DELL T40 and DELL R250 servers with PROXMOX and a ZFS RAID1 to host 2 virtual machines, one with WINDOWS SERVER AND SQL SERVER, and another to share files via Windows due to the requirements of the application suite.

This means that each server has the following characteristics:

*DELL T40 / DELL R250
*PROXMOX 7.3-3 / 7.2-3.
*32GB / 64GB RAM.
*2 SPCC SSD DISK 1TB STORAGE.
*1 VM WITH WINDOWS SERVER 2022 AND SQL SERVER 2019.
*1 VM WITH WINDOWS SERVER 2022 FOR SHARING FILES.
*BACKUPS OF THE VIRTUAL MACHINES IN THE SAME RAID1 ZFS CONFIGURED TO RUN EVERY DAY AT 12AM.

It is currently impossible to run backups of virtual machines locally, they take too long, and the IO delay goes way up to 90%. Small virtual machines of 50GB take up to more than 1 hour to complete, also the restoration takes too long...

Once the backups reach a limit that they cannot be executed since they give an error. The general performance of the Equipment is degraded and that of the virtual machines. So much so that write speeds reach lows of 7MB/s and 10MB/s.

I have increased the ARC even to consume half of the available RAM, as it comes by default. Since I modified it to use 8GB in the DELL T40 and 12GB in the DELL R250, which is more than enough. Even so, the problem persists, unfortunately.

The question is, what can I do to deal with this? I'm not entirely sure these SSDs are enterprise SSDs. Anything I can try to make performance more manageable while locating enterprise grade SSDs? I think ZFS is killing off SSDs pretty fast too.

We already requested disks for external backups of the VMs. Because if they are made local to the ZFS RAID they take too long and performance is affected.

-----------------------------

I have a TRUENAS server installed in a RAID 1 ZFS (2 SSDs for OS) and 3 4TB HDDs in RAIDZ1 (File Storage), with no performance issues of any kind. Is this a good comparison?

 

[VDNKH]

The performance and speed of the backups were initially excellent, without any impact and they were done very fast.

We even had a backup DELL R250 in case of emergencies with a virtual machine running Windows Server 2022 and SQL ready to host the databases in case of an emergency.

The VM turned off and automatic backups every night. Even without workload the performance drops off sharply in about 30 days.

 

[LnxBil]

The question is, what can I do to deal with this? I'm not entirely sure these SSDs are enterprise SSDs. Anything I can try to make performance more manageable while locating enterprise grade SSDs? I think ZFS is killing off SSDs pretty fast too.

You have to answer this. A server should have caddies, so unplug the disks while the server is off and look at the label.

 

[VDNKH]

You have to answer this. A server should have caddies, so unplug the disks while the server is off and look at the label.

I'm almost 100% sure it's not an enterprise SSD.
Is it behaving like this because of ZFS and its copy-on-write?

 

[_gabriel]

The ZFS on consumer SSD topic of the week !
Only a warning directly from the PVE installer can prevent flood forum.
You can read the one from last week.

ZFS require enterprise ssd with at least power protection.
Regular ext4 (lvmthin will used for vm disks), will be faster but less secure in case of power failure or crash. Like regular desktop/laptop.

 

[VDNKH]

Any way to increase performance while getting the enterprise SSDs?

 

[_gabriel]

it's possible SSD are already burned, check their "smart" status, written counter.
I'll do a test of one disk on another PC (so ZFS mirror will be broken) , format disk then do a write test ( fio for 10 minutes or large copy file).
if speed is "regular" you can do the slow and safe process of backup/restore VMs restore on a regular ext4 install.
sorry for my english ...