Proxmox 5.2 on HP DL380 G9 with P440ar and SAS drives - final recommendation?

[keson]

Dear gurus!
I am running a ML380 G9 server with SATA drives and Proxmox VE 5.1 and love the performance and reliability.
Recently we got a new HP DL380 G9 server with integrated HW RAID Smart Array P440 and 6 SAS 15k drives.

I want to move my setup to this more equipped server, but I am struggling with all the same issues many here described. Unfortunately I did not get a clear conclusion on how the situation is as of June 2018 when I want to use ZFS.

It is clear through all the threads here to NOT to use HW RAID for ZFS as the fs needs direct access to disks.

First I saw here that it shall work, so I gave it try:
https://forum.proxmox.com/threads/installing-proxmox-4-on-a-new-hp-dl-380-g9.32380/

Unfortunately, I have experienced the same behaviour when installing the Proxmox as many others here. I have switched the controller to HBA mode and so I can see the disks during installation, all seems fine, but it never boots, it always ends in GRUB RESCUE.

I have tested to remove ALL disks but one and install the Proxmox to just one disk. That seemed to work. I was even able to plug in the other disks once it was up and running and create a new pool in ZFS, but as soon as I restarted the server, no boot, just grub rescue.

Then I saw some good news on using the ACHI connected disks on this setup,
https://forum.proxmox.com/threads/zfs-disk-layout-for-new-install.29607/#post-186411
but I was unable to make this work - did not find any BIOS configuration on how to use it.

Also here a raid0+1 is recommended, but I suppose the user will loose ZFS:
https://forum.proxmox.com/threads/hp-proliant-ml-350-g9-best-practice.22103/#post-111749

Another thread proposes using some special drivers:
https://forum.proxmox.com/threads/cant-install-proxmox-on-hp-proliant-dl380-gen9.20736/#post-107031
And then further some other attempt:
https://forum.proxmox.com/threads/cant-install-proxmox-on-hp-proliant-dl380-gen9.20736/#post-152658

But at the end of the day, I do not see any SUCCESS story. So what would you do with HP G9 and p440ar controller and SAS disks? Is there a way to install proxmox and ZFS there or do I need another HW?

Final thought.
On the older ML server I use BIOS legacy boot options and it works I was unable to make the UEFI to boot. The same seems the issue on the DL 380 - whatever I did try, never except UEFI. What is the best practice for LEGACY BIOS / UEF on HP servers with proxmox? Is there any real reason to fight for UEFI?

One more thought...
Friend of mine suggests to remove the Smart Array p440ar and buy a simple HBA controller, but I see many which still have "raid" in their name or battery or cache in the features and I have red here, that any such controller shall be avoided. What is the good value simple HBA for use in HP server and 6 SAS 15k drives? Would this work: Supermicro HBA AOC-S3008L-L8e ? Or any other?

 

[alexskysilk]

But at the end of the day, I do not see any SUCCESS story. So what would you do with HP G9 and p440ar controller and SAS disks? Is there a way to install proxmox and ZFS there or do I need another HW?

For starters, just because this particular hardware combination isnt suitable for zfs doesnt mean that you can't use it with LVM. There is no question that ZFS offers more features but LVM is functional, performant, and will work with your raid controller without issue.

HPs really prefer HP branded HBAs/controllers and will not expose RAID bios during boot for non HP devices. just food for thought if you were to go with hp. If you get an HBA, get an H240.

 

[keson]

For starters, just because this particular hardware combination isnt suitable for zfs doesnt mean that you can't use it with LVM. There is no question that ZFS offers more features but LVM is functional, performant, and will work with your raid controller without issue.

HPs really prefer HP branded HBAs/controllers and will not expose RAID bios during boot for non HP devices. just food for thought if you were to go with hp. If you get an HBA, get an H240.

Thanks Alex. Somehow I felt in love with ZFS, but you are right, it is not a show stopper to not to have it. Just wanted to see it more and more in action. I really appreciate your answer. This is exactly what I needed to hear. Will check the H240. I did not even expect any existing HP controller suitable for this. I can exchange it for the current P440 with anyone perhaps
THANKS!

 

[_Adrian_]

Hey @keson

Which way did you end up going ??
I'm just about done "building" mine ( local IT company was liquidating a bare bones 380 G9 for $20 ) and just waiting on the final pieces to arrive...

 

[keson]

A lot of time passed but stil those servers are in use, now with proxmox 8. And i have 6 such servers - G7-G9 running… so far so good.
The answer is rather simple.
For one server - the main one - i got HBA controller and replaced the delivered hw raid.
But for all others i was not able to do so, so i went against the recommendations. But worth mentioning, those servers are in a cluster and are used mostly for replicas - to be used just in case. And to make sure the underlying hw raid and its disks are ok, i use the ilo and snmp to monitor their health - librenms.
It is generally not recommended as zfs does not have direct contact with the disks and cannot react on hw failures, but it works. Also the hw raid is redundant (mirrors) so when one disk fails, the raid keeps working and ilo reports back so i have enough time to fix.

I would conclude that such setup is good for our purpose, i would not use it as a standalone mission critical server though. But in a hyper convergent setup with multiple nodes and cephs for example where whole node and its storage can fail and the cluster still works - namely if such hw is obtained as depreciated - i would not be afraid to use it.
In past 8 years no failure.
And now the community expells me