Move or not move Proxmox VE boot to SSD on existing system upgrade?

[ggallo]

A client of us (a school) built 8 identical Proxmox VE servers with only 2 HDDs in each (in a ZFS mirror). The preformance is ... slow. They decided to buy additional 2x SSDs to all 8 servers now.

We need to help them upgrade the machines. My question is that which method is better for long term:
- add the 2x SSDs as a new pool in ZFS mirror and move essential VM disks to those, keep Proxmox boot and large VM data disks on HDD mirror pool
- move Proxmox VE to the SSD mirror pool to boot from, put essential VM disks on it and keep large data VM disks on HDD mirror pool

If the latest is the better in any way in the long term, which is the best method for the move?
- ZFS send/receive from the current HDD pool
- new install, the copy /etc/pve and VMs

Thanks for any advice!

 

[JensF]

#1 is the way I would go. If you have enough slots and ports for the new SSDs.
I hope these are enterprise grade SSDs for ZFS!
Just add them to the server, create a new pool and move the VM disks over.

 

[LnxBil]

As always, I recommend to add the SSDs to the pool as special devices, which will yield a faster system without the additional overhead of having two pools sharing the ARC etc.

https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_special_device

With this, you can split the metadata from the actual data, which will speed up everything. You can also add another dataset with a special_small_blocks=128K and add it to PVE as another storage. All data will be written to the SSD instead of the HDD for "needier VMs".

Try this out in a VM to familiarize yourself with everything and monitor the data splitting.

 

[louie1961]

You will get no benefit from putting the boot partition on an SSD drive. You may also want to investigate faster storage solutions instead of keeping VM data on a HDD mirror. Almost nothing could be slower. I really prefer to keep as much of my data on a separate NAS as possible.

And in my opinion, whether you store VMs on them or use them as special devices, if you are using SSDs without power loss protection, you are asking for trouble. Most consumer grade drives lack power loss protection. As noted above, definitely go for enterprise grade drives.

 

[ggallo]

#1 is the way I would go. If you have enough slots and ports for the new SSDs.
I hope these are enterprise grade SSDs for ZFS!
Just add them to the server, create a new pool and move the VM disks over.

I likely step on this road, as this is the easiest and fastest to implement. Thank you!

 

[ggallo]

As always, I recommend to add the SSDs to the pool as special devices, which will yield a faster system without the additional overhead of having two pools sharing the ARC etc.

https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_special_device

With this, you can split the metadata from the actual data, which will speed up everything. You can also add another dataset with a special_small_blocks=128K and add it to PVE as another storage. All data will be written to the SSD instead of the HDD for "needier VMs".

Try this out in a VM to familiarize yourself with everything and monitor the data splitting.

I don't think that 2 mirrored HDDs benefit much from SSD special devices in this case. If I put small blocks on SSD to speed up VMs, I create a relative complicated ZFS pool with little to none benefit compared to a separate SSD mirror pool.
But, thank you for your suggestion!

 

[ggallo]

You will get no benefit from putting the boot partition on an SSD drive. You may also want to investigate faster storage solutions instead of keeping VM data on a HDD mirror. Almost nothing could be slower. I really prefer to keep as much of my data on a separate NAS as possible.

And in my opinion, whether you store VMs on them or use them as special devices, if you are using SSDs without power loss protection, you are asking for trouble. Most consumer grade drives lack power loss protection. As noted above, definitely go for enterprise grade drives.

The 8 servers gone to 8 separate schools which do not have other server side components (like a NAS for bulk storage), and the school central is poor, cannot afford much more hardware purchase. This is also why the HDD mirror build in the first place instead of (server grade) SSDs.
As I informed, the SSDs they bought is server-grade, with PLP and write endurance >1.0 DWPD.
So I willing to put them in a separate mirror pool and move VM's system virtual disks to the new SSD pool to speed up them. The file server shares will remain on the slow HDDs, but this is not a big problem for them, they said.
Thank you for your help!

 

[JensF]

No other components? How do you do backups?

 

[ggallo]

No other components? How do you do backups?

Backups done by Proxmox Backup Server running on a Synology Plus series NAS (with HDDs in it). Of course they cannot store live data on the same NAS, and cannot afford another NAS yet.