Added 2 new storage drives to proxmox build, lost ability to boot proxmox

[ryantboldt]

I'm at work now away from my proxmox box so I may not be able to provide answers to questions immediately, but I figured I'd get this thread rolling now so perhaps I could have something to look at when I get home. My proxmox box has been working fine for months. Last night I wanted to add 2 8tb hard drives to the chassis so that I could add storage to a vm of mine. I gracefully shut down the box, added the two drives, went to power on the box and got nothing.

This box has been running without monitor for several months now so I didn't immediately know what was wrong. I disconnected the two drives and booted again and this time things booted successfully. I shut down once again and figured it may be something with the boot order... spent 30 minutes hunting down a vga monitor and powered back up.

After this I was never again able to get proxmox to boot successfully. Every time I power on now with or without the 2 additional drives added, I simply end up at a black screen with blinking cursor.

So my original configuration was 4 samsung ssds in a (from my memory raidz 10) zfs connected to an IMB Serveraid M1015. To be fair I'm not to familiar how a zfs raid 10 boots (with regards to a boot device) (ie, which of the 4 drives would you select to boot from) Last night I noted that my motherboard would only let me select lun0 samsung as a bootable device.

So as it stands... working on this box until 1 am, I tried booting to the proxmox installer usb and tried recovery it said compression algorithm inherit not supported and no boot found (the no boot found is from my memory and actual wording was likely different.)

Thank you for any help you may provide.

 

[lhorace]

Hello there,

Does it even POST?

Regards

 

[ryantboldt]

Hello there,

Does it even POST?

Regards

the hardware seems fully functional. i have full access to the bios and can boot the proxmox installer. Its when I try to let it boot its previous proxmox installation via the 4 ssds in zpool does it simply go to a black screen with blinking cursor.

 

[ryantboldt]

I somehow managed to get proxmox to boot and now I see the status of the zpool.... I feel like it didn't used to look like this... is it alarming to see sdd and sda and then ata and ata?

Why are 2 of the drives showing their information and the other 2 aren't?

scan: scrub repaired 0B in 0h21m with 0 errors on Sun Jun 9 00:45:25 2019
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdd2 ONLINE 0 0 0
sda2 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
ata-Samsung_SSD_860_EVO_500GB_S3Z1NB0XXXXXXX ONLINE 0 0 0
ata-Samsung_SSD_860_EVO_500GB_S3Z1NB0XXXXXXX ONLINE 0 0 0

errors: No known data errors

 

[mbosma]

It seems you have added the two disks on id instead of its /dev/sdX name. This should not be a problem.

Did you boot your PVE normally or with a recovery cd?

 

[ryantboldt]

It seems you have added the two disks on id instead of its /dev/sdX name. This should not be a problem.

Did you boot your PVE normally or with a recovery cd?

I was able to boot PVE normally after re-arranging / reconnecting / resetting my drives and lsi card. Im curious as to what would cause 2 of the drives to use id vs not as the zpool was configured via the Proxmox installer gui.

I'm going to have to look at any downsides to having id vs non id in the pool and see if I need to attempt to correct this.

 

[ryantboldt]

i guess its just impossible to add hard drives to a proxmox system after it's initially configured. i can boot the pve fine all day long now. the second i connect any hard drive additionally to the system I get a grub rescue

no such device unknown filesystem entering rescue mode grub rescue

 

[ryantboldt]

i must be on the truman show. 6 plus hours and a fresh install of proxmox with the additional drives pre-installed and can't actually boot with anything but the root zfs zpool attached. change sata ports change sata controllers change root zpool to zfs by id.

 

[ryantboldt]

well another 3 hours of troubleshooting resulted in recreating the root zpool by detaching and reattaching the drives one at a time to use the by-id instead of /sdx. I was then finally able to add new drives to the system and create a second zpool. I kept reading that having a root zpool with drives identified by /sdx didn't matter but it was only until i made the zpool look for the id that it was able to boot alongside new hardware.

 

[mbosma]

I'm sorry for my late reply.
I initially installed my pve on a dual mirror pool with the sdx format and changed them over to by-id.
Because of some janky hardware I used in the past some zpools would break due to the disks suddenly switching sdx identifiers after reboot.

In my experience that way of creating your zpool (or changing it to that format) is the most stable.