proxmox on zfs fails booting

[AxelTwin]

Hi everyone,
I installed proxmox on zfs on a dedicated server at hetzner, but I cannot manage to boot.
I can start it using rescue mode and loading it from vnc, but if I try to boot in normal mode the server get stuck on bios menu.
This happens only on zfs system.
Anyone has an idea ?

 

[LnxBil]

Does your hetzner server uses UEFI or BIOS boot?

 

[AxelTwin]

It uses UEFI
is that my probem ?

system currently booted with legacy bios

 

[Dunuin]

Then you should enable CSM in UEFI if your disks require to boot with legacy BIOS.

 

[AxelTwin]

Ok, I will try that then.

Anything I should do from this proxmox wiki:

Setting up a new partition for use as synced ESP
To format and initialize a partition as synced ESP, e.g., after replacing a failed vdev in an rpool, or when converting an existing system that pre-dates the sync mechanism, proxmox-boot-tool from pve-kernel-helpers can be used.

the format command will format the <partition>, make sure to pass in the right device/partition!

For example, to format an empty partition /dev/sda2 as ESP, run the following:

# proxmox-boot-tool format /dev/sda2

To setup an existing, unmounted ESP located on /dev/sda2 for inclusion in Proxmox VE’s kernel update synchronization mechanism, use the following:

# proxmox-boot-tool init /dev/sda2

Afterwards /etc/kernel/proxmox-boot-uuids should contain a new line with the UUID of the newly added partition. The init command will also automatically trigger a refresh of all configured ESPs.

Updating the configuration on all ESPs
To copy and configure all bootable kernels and keep all ESPs listed in /etc/kernel/proxmox-boot-uuids in sync you just need to run:
# proxmox-boot-tool refresh

 

[AxelTwin]

Then you should enable CSM in UEFI if your disks require to boot with legacy BIOS.

Aouch ! option is greyed out in bios...

 

[Dunuin]

Then you need to boot PVE with systemd/ESP instead of grub.

 

[AxelTwin]

Then you need to boot PVE with systemd/ESP instead of grub.

I was looking at that on the wiki. not completely sure how to achieve that. any chances to choose that option at installation ? could you elaborate a bit on what should be the setup ?

 

[Dunuin]

As far as I understand it depends on the system your are installing PVE to. When using BIOS or UEFI with CSM it will install the grub bootloader. When using UEFI without CSM it will install systemd bootloader.

 

[AxelTwin]

As far as I understand it depends on the system your are installing PVE to. When using BIOS or UEFI with CSM it will install the grub bootloader. When using UEFI without CSM it will install systemd bootloader.

Yeah, for me no choice, cant enable CSM so proxmox installed with grub. I can see there is a way to switch to systemd bootloader but it's confusing me right now

 

[LnxBil]

Yeah, for me no choice, cant enable CSM so proxmox installed with grub. I can see there is a way to switch to systemd bootloader but it's confusing me right now

Just reinstall. It is HARD to switch from one to the other.

How did you install it in the first place? I normally do a local install in PVE with two disks with the settings from the Hetzner setup and copy the disk images to the hetzner disks.

 

[AxelTwin]

sorry, didn't see your reply.
I am trying to install root system on zfs.
I managed to get the installer to run to 99% with kvm console and usb bootable iso.
It fails at writting the systemd-bootloader.
proxmox-boot-tool init /dev/nvme0n1p2 gives: Failed to set LoaderSystemToken EFI variable: Invalid argument

(sorry for screeshots, cant copy/paste from kvm)

 

[LnxBil]

That may be something special with NVMe booting I'm not aware of. Sorry.

 

[itsit_cc]

Hi @AxelTwin,
I think you will find the solution to your problem if you have a look here:
https://github.com/ariadata/proxmox-hetzner
Basically what it says is that when installing Proxmox on ZFS with HW that only supports UEFI you have to use QEMU to boot with UEFI firmware adding the flag "-bios /usr/share/ovmf/OVMF.fd"...
We have done that and it works.
Regards,