Proxmox VE on ZFS USB Drive mirror with ZIL on SSD

Dec 29, 2016
2
0
21
38
Hi,

I'm running a 5 node pve Cluster with HP DL360 gen8 Servers since about 3 years. The integrated HP P420i RAID Controller does not allow booting from disks when in HBA mode, so I ran 6 of the SFF disks as separate raid0 each for ceph osds and 2 of the SFF disks as raid1 for pve boot disks. Basically this is working but monitoring disks and changing failed hdds is a pain in the ass.

Going from P420i Hardware RAID to HBA mode - boot from usb drive(s):
Now I'm thinking of installing pve on ZFS mirrored usb drives (2 drive mirror with 1 spare) with a SSD partition as ZIL device. I know that these USB drives are not known as reliable root filesystem storage due to bad write performance but with the ZIL all writes are going to ssd and are synced back (as combined writes) to the usb drives periodically (time range configurable). Reads are then coming from the ARC (RAM) or optionally L2ARC (SSD).

What do you think about this setup for a pve production environment?

Things to know about HP DL360 gen8 Servers:
again... boot from single disk on P420i in HBA mode is not working...
A separate PCIe NVME SSD as boot device is no option because it is not working either...
 
Now I'm thinking of installing pve on ZFS mirrored usb drives (2 drive mirror with 1 spare) with a SSD partition as ZIL device. I know that these USB drives are not known as reliable root filesystem storage due to bad write performance but with the ZIL all writes are going to ssd and are synced back (as combined writes) to the usb drives periodically (time range configurable). Reads are then coming from the ARC (RAM) or optionally L2ARC (SSD).

A better ideea is to buy a cheap sata controller(even PCI) and then to add 2 HDDs. With usb HDD after 5 sec(default) even ZIL data will be flush'ed to disks as the ARC data, so in the end you will have some big bursts of data on the usb-hdd. Another problem, is the fact that the HDD-usb suspend could also create problems!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!