How to install Proxmox on PCIe NVMe disk when bios can't boot from it? SD card?

[morph027]

Jip. You can skip the copy of /boot, grub-install should take care of this.

http://jarrodla.blogspot.de/2014/04/hp-microserver-gen8-boot-from-5th-sata.html (Mind the devices, might be something like /dev/mmcblk0p1 or so)

 

[SanderM]

I just reinstalled proxmox and booted into the resque environment. Created new partition on SD card of 128mb, formatted it using ext2.
Did not mount it. Then I did "grub-install /dev/sda" and it reported no errors.
But if I mount /dev/sda1 it doesn't contain anything. Should I copy /boot myself then?

And where do I place the chainloader grub.conf? It should be in /mnt (where /dev/sda1 is mounted)/grub/grub.conf I suppose?

 

[morph027]

Are you sure that sda is the sdcard? If you, mount the card before using grub-install.

The --boot-directory= flag for grub really makes sure all neccessary modules and so on are on the mounted card.

mount /dev/sda1 /mnt
mkdir /mnt/boot
grub-install --boot-directory=/mnt/boot /dev/sda

Now, just copy the chainloader grub.cfg into /mnt/boot/grub/

 

[SanderM]

Ok. I did what you said but then it boots straight into grub with a bash like shell.
If I type "ls" there it shows:

grub> ls
(hd0) (hd0,msdos2) (hd0,msdos1)
grub>

So, hd0 seems to be the SD card. (I create two partitions on the SD card.)

But it's not loading proxmox

 

[SanderM]

Could it be that grub doesn't recognize the NVMe PCi-e SSD either and needs the kernel on the SD card in order to boot?

I find it strange that the "ls" command is only showing the SD card.

 

[morph027]

Jip, just thought the same...

 

[SanderM]

Okay; So the only way to fix it is by putting the kernel on de SD card.

But the kernel isn't reading/writing itself from there once it's loaded, right? So it shouldn't be much of a problem to have it on the SD card?

 

[fabian]

if your bios cannot boot from a device, then grub often also does not "see" it - that's why I recommended putting the full grub on the SD card. in that case, the initrd/kernel are the first things that need to see the root device, and if they don't, than you have a bigger problem then not being able to boot from that disk

 

[SanderM]

Okay. I will reinstall it like that

Thanks all!

 

[morph027]

Ok...damn it, grub

Please keep us posted, i'm planning this later too. But i hope that the BIOS/UEFI then might be able to boot from NVMe directly

 

[SanderM]

I reinstalled with everyting from /boot on the SD card and now it's working again.

 

[morph027]

Nice. Once i get my hands on any nvme device, i'm going to try, if the grub-efi from jessie-backports will work.

Just in case, if someone want's to lend me hardware to try, pm me

 

[proxtest]

I boot a cluster from USB-Stick's. It's easy to dd the stick to another stick, so u always have a backup if something goes wrong with a kernel update. The sticks now 2 years old and still working flawless!

 

[fabian]

Nice. Once i get my hands on any nvme device, i'm going to try, if the grub-efi from jessie-backports will work.

Just in case, if someone want's to lend me hardware to try, pm me

In case you want to (and unless they haven't hit the regular repositories already until then), I can provide PVE-compatible grub packages based on Debian testing (Grub 2.02 beta3) to you. Just ping me in this thread, and I'll put them somewhere public.

 

[morph027]

Oh, sure. I'll let you kown, when i have the hardware ready. Thanks.

 

[ms-pve]

adapted for ZFS root partition, as grub rescue might not see the ZFS root: (e.g. compression algorithm not found..)

don’t reboot after setup (checkbox before install starts), then abort setup
zpool import -R /mnt rpool -d /dev/disk/by-id/…-part3

plug in boot usb-stick
format usb with msdos partition table
create 512MB ext2 partition, set bootable flag
take note of usb-partition-UUID

mount -t proc /proc /mnt/proc
mount --rbind /dev /mnt/dev
mount --rbind /sys /mnt/sys

chroot /mnt /bin/bash
source /etc/profile
fstab add: UUID=[usb-partition-UUID] /boot auto nosuid,nodev,nofail 0 0
cp -r /boot /boot.old
mount /boot
cp -r /boot.old/* /boot/
update-grub2
update-initramfs -u
grub-install /dev/sd[usb]
umount /boot
exit

zpool export rpool

umount /mnt/sys
umount /mnt/dev
umount /mnt/proc

reboot

 

[janssensm]

@gabo14 thanks for writing down these steps.
The only option left for my older system was to boot from USB and these steps solved it for me.

I would like to ad to your instructions that I needed to boot in the Debug mode option, the normal install mode seems to reboot the system when selecting abort.
And the unmounting steps won't succeed all, but this appears not to be an issue as settings are saved and the systems boots fine from usb now.

 

[vdias]

Careful, abort setup = reboot... and then we need to start all over... better jump to terminal using CTRL + ALT + F3.

first mount command not correct...

"zpool import -R /mnt rpool -d /dev/disk/by-id/…-part3"
Cannot import 'rpool": no such pool avaiable

I'm using 7.2 iso... using zpool list... no pools avaiable.

I think that proxmox default instalation do not create any ZFS pool... just a EXT4 partiton...