HP Gen9 ZFS Boot Smartarray 440ar

[wimp9849]

Hi, i need advice on the best setup for the hardware i have.

• HP DL360 Gen9
• 64GB Ram
• 2x300GB SAS 15K
• 6x600GB SAS 15K

My plan is to use zfs on the 6x600 GB in Raid 10 for a total of 1.8TB usable and the 2x300GB for mirroring the proxmox installation.

What's the best way? Install using zfs mirror on the first 2 disk and create the pool for the vms after install? Or is better to not use ZFS for the install disks?
if so, how should I proceed?

Thanks

 

[wolfgang]

Hi,

IMHO your first guess is plan is good.

 

[wimp9849]

so, i'll go with 2 pool ZFS.

Thanks

 

[wimp9849]

I've tried installing on the first 2 disk, everything is fine but when i reboot it doesn't boot.

Legacy mode active, order bios is ok to boot first from HP 440ar, Hp400ar is in HBA.

I've tried to install with only the 2 disk installed and it boot correctly, but if add the others disk it doesn't boot

 

[alexskysilk]

I've tried to install with only the 2 disk installed and it boot correctly, but if add the others disk it doesn't boot

"It doesnt work" is rarely enough information for troubleshooting. just sayin.

 

[wimp9849]

"It doesnt work" is rarely enough information for troubleshooting. just sayin.

Can you elaborate more? Doesn't Work because of hp card? because of 2 different pool? Or because of zfs?

So If the problem is 2 different pool i can discard the 2 disk for the os and Just use the single pool with the bigger drive for install and vm Image.

If the problem is zfs for boot i can install on a single disk ext4, but If the disk fail i have to reinstall proxmox and recreate the macchine (at Least the configuration)

 

[alexskysilk]

For starters, is there an error message?

 

[wimp9849]

For starters, is there an error message?

No. It Just not boot, it pass all the boot order until It reach the last option(network boot). Is like It is missing where to boot. But if i remove all the disk except for the 2 with the os it boot. I have no option on boot order. I have cd,USB, ecc and hp smartarray. But inside smartarray i have not a boot order

 

[alexskysilk]

No. It Just not boot, it pass all the boot order until It reach the last option(network boot)

So this isn't a proxmox issue at all. Can you manually select a boot device, and if so, is your proxmox volume available to be selected?

 

[wimp9849]

So this isn't a proxmox issue at all. Can you manually select a boot device, and if so, is your proxmox volume available to be selected?

But if i remove all other disk It boot.

Edit: i haven't find an order inside the controller. I can choose cd,USB, internal sata, hp smartarray, network boot

 

[alexskysilk]

good. what happens when you select "hp smartarray"?

 

[wimp9849]

good. what happens when you select "hp smartarray"?

If i have only the 2 os disk It boot.
If i have the 2 os disk (in the First 2 bays) and the other 6 disk It doesn't boot

 

[wimp9849]

1 more hint. Tried installing with only 6x600gb using zfs raid10. The server start in grub command line with the error

Unknown device 1
Entering rescue mode
Grub rescue>

 

[fabian]

1 more hint. Tried installing with only 6x600gb using zfs raid10. The server start in grub command line with the error

Unknown device 1
Entering rescue mode
Grub rescue>

that probably means that your smart array does not present all disks as bootable. since it works with only two, it might be that the presented disks get moved around when adding more. you can try switching the disks around (or if you don't yet have data on them, wipe them and try re-installing and selecting different disks for the OS in the installer, wiping between attempts to prevent confusion)

 

[wimp9849]

that probably means that your smart array does not present all disks as bootable. since it works with only two, it might be that the presented disks get moved around when adding more. you can try switching the disks around (or if you don't yet have data on them, wipe them and try re-installing and selecting different disks for the OS in the installer, wiping between attempts to prevent confusion)

I have installed on 6 disks choosing zfs raid 10. Isn't the boot partition installed on all disks?

 

[fabian]

I have installed on 6 disks choosing zfs raid 10. Isn't the boot partition installed on all disks?

yes, but you need at least the right three disks (one of each mirrored pair) in your scenario. you can check which/how many disks grub sees with "ls" in the grub rescue shell

 

[wimp9849]

yes, but you need at least the right three disks (one of each mirrored pair) in your scenario. you can check which/how many disks grub sees with "ls" in the grub rescue shell

I've attacched what i see know using 6 Disk in raid10

 

[fabian]

I've attacched what i see know using 6 Disk in raid10

so it only passes one disk (the one with partitions) and a lot of garbage (the ones without)..

 

[wimp9849]

so it only passes one disk (the one with partitions) and a lot of garbage (the ones without)..

Ok!

So, is there any chances to use Proxmox on this server or do i have to give up?

 

[alexskysilk]

Ok!

So, is there any chances to use Proxmox on this server or do i have to give up?

I appreciate that this is frustrating, but you have two choices whenever deploying any form of solution- pay for it or learn how to do it yourself. all your questions are answerable by googling generally and searching this forum specifically. If you are unable/unwilling to research and give up, I doubt anyone here will be offended.

IF you want to boot from a hardware raid volume, you have to specify it as the booting lun in your controller bios. you can start here: http://h10032.www1.hp.com/ctg/Manual/c02289065.pdf

IF you want to boom from a zfs lun and are getting stuck in grub, you may wish to consult the Proxmox documentation, which is pretty good- have a look here: https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Grub_boot_ZFS_problem