Proxmox on ZFS at hetzner

[AxelTwin]

Hi everyone,
did someone managed recently to install proxmox on zfs on a hetzner server ?
I used to do it several times in the past but could not manage to do it anymore.
Tried with a bootable iso installed by hetzner's tech but it brakes the network and I loose connectivity.
Tried from vnc in rescue mode but didn't work neither.
I would appreciate if someone has an idea

 

[Moayad]

Hi,

If you post what occurs exactly and where broken, and what message you see, etc... that will help us to identify what happens exactly. Another question, which Proxmox VE ISO version are you trying to install?

Have you tried to send to Hetzner support in order to check for the Hardware on your server?

 

[AxelTwin]

Hi,
Thanks for replying.
I am trying to install PVE-7.4.
Hetzner doesn't hvae proxmox ZFS image so you c'an't install proxmox on a ZFS pool.
I installed it from rescue mode with VNC so I get the GUI and can install from that but the installer pick the wrong interface, it sould be something like enp5s0 but the only choice is enp5, I guess it sees the rescue server's interface name.
So once you reboot in normal mode, it looses connectivity and I can't access proxmox.
If I get back in rescue mode, I can't mount zfs system to access /etc/network/intefaces and change the config.
I managd to have a KVM console plugged on the server and boot on a usb key with proxmox iso on it but same story, as soon as the installer configure the NIC it looses connectivity.

 

[AxelTwin]

Ok, so the problems seems not related to networking.
The server won't boot after intall as it doesn't detect the disks once in zfs.
I will open a new thread for that.
Thanks

 

[Moayad]

Regarding the network configuration, you can know the real name of the NIC using the following command:

udevadm info -q all -p /sys/class/net/<Adapter Name> | grep ID_NET_NAME

After you know the real NIC name, you can edit the /etc/network/interfaces file. Don't forget to do a `ifreload -a` command, and see if that will fix the issue.

Regarding the ZFS can you provide us with the Syslogs?

 

[AxelTwin]

Regarding the network configuration, you can know the real name of the NIC using the following command:

udevadm info -q all -p /sys/class/net/<Adapter Name> | grep ID_NET_NAME

After you know the real NIC name, you can edit the /etc/network/interfaces file. Don't forget to do a `ifreload -a` command, and see if that will fix the issue.

Regarding the ZFS can you provide us with the Syslogs?

I have identified the issue with ZFS but couldn't manage to fix it yet.
My server is UEFI only (enabling CSM is greyed out in bios) and the server tries to boot from legacy.
I am trying to find a way to rewrite boot sector to swap to systemd bootloader.
I read several tuto but didn't find anything very clear, If you have an idea I'll take it

 

[itsit_cc]

Hi @AxelTwin,
I think you will find the solution to your problem if you have a look here:
https://github.com/ariadata/proxmox-hetzner
Basically what it says is that when installing Proxmox on ZFS with HW that only supports UEFI you have to use QEMU to boot with UEFI firmware adding the flag "-bios /usr/share/ovmf/OVMF.fd"...
We have done that and it works.
Regards,

 

[matze1]

Hi @AxelTwin,
I think you will find the solution to your problem if you have a look here:
https://github.com/ariadata/proxmox-hetzner
Basically what it says is that when installing Proxmox on ZFS with HW that only supports UEFI you have to use QEMU to boot with UEFI firmware adding the flag "-bios /usr/share/ovmf/OVMF.fd"...
We have done that and it works.
Regards,

Hi,

i have the same problem.
Where and when should i add the flag "-bios /usr/share/ovmf/OVMF.fd" ?

Regards,

Matze

 

[AxelTwin]

Hi,

i have the same problem.
Where and when should i add the flag "-bios /usr/share/ovmf/OVMF.fd" ?

Regards,

Matze

You should add the flag when seting up installation through VNC, Command: #### If UEFI Supported

#### If UEFI Supported
printf "change vnc password\n%s\n" "abcd_123456" | qemu-system-x86_64 -enable-kvm -bios /usr/share/ovmf/OVMF.fd -cpu host -smp 4 -m 4096 -boot d -cdrom ./pve.iso -drive file=/dev/nvme0n1,format=raw,media=disk,if=virtio -drive file=/dev/nvme1n1,format=raw,media=disk,if=virtio -vnc :0,password -monitor stdio -no-reboot

#### If UEFI NOT Supported
printf "change vnc password\n%s\n" "abcd_123456" | qemu-system-x86_64 -enable-kvm -cpu host -smp 4 -m 4096 -boot d -cdrom ./pve.iso -drive file=/dev/nvme0n1,format=raw,media=disk,if=virtio -drive file=/dev/nvme1n1,format=raw,media=disk,if=virtio -vnc :0,password -monitor stdio -no-reboot

 

[chrcoluk]

Fighting this right now, I have done the install via qemu using my normal method, but the boot fails at post (looking via KVM) as if there is no OS installed, however it will boot fine inside a qemu instance inside rescue os.

--

Fixed it via setting storage oprom to UEFI only in bios. So seems was a UEFI bios issue. Going to make sure I havent broken Hetzner's rescue OS process before I hand back the KVM.

Looks like hetzners network boot (which they use for rescue OS and other stuff) does not support UEFI boot from local disk.

So its fixable, but it involves editing bios in a way that skips their network boot, and then can boot bare metal from UEFI, the downside is that whenever need to do some kind of rescue will need KVM.

They have a UEFI rescue OS option as detailed here, I have requested it.

https://docs.hetzner.com/robot/dedicated-server/operating-systems/uefi/

--

All good now Hetzner activated the UEFI boot script.

 

[AxelTwin]

Fighting this right now, I have done the install via qemu using my normal method, but the boot fails at post (looking via KVM) as if there is no OS installed, however it will boot fine inside a qemu instance inside rescue os.

--

Fixed it via setting storage oprom to UEFI only in bios. So seems was a UEFI bios issue. Going to make sure I havent broken Hetzner's rescue OS process before I hand back the KVM.

Looks like hetzners network boot (which they use for rescue OS and other stuff) does not support UEFI boot from local disk.

So its fixable, but it involves editing bios in a way that skips their network boot, and then can boot bare metal from UEFI, the downside is that whenever need to do some kind of rescue will need KVM.

They have a UEFI rescue OS option as detailed here, I have requested it.

https://docs.hetzner.com/robot/dedicated-server/operating-systems/uefi/

--

All good now Hetzner activated the UEFI boot script.

Good job !

 

[ronaldst]

I had some issues with this a while back on an AX101.

The easiest and best solution when you want full control of the setup procedure is to ask for a IPMI/Remote Console. You can do this in the support tab of Hetzner Robot. You can request staff to download and setup a bootable USB stick for you, just give them a link to the ISO you want them to prepare for you in the same support ticket.