[PVE-9 BETA] PVE Nested Upgrade from 8 to 9 breaks boot

[jsterr]

I just tested the PVE-8 to PVE-9 Upgrade on a freshly installed PVE-8 Installation (Nested, VM) so not on physical hardware. After upgrading and rebooting, the VM does not boot anymore and goes to VM-BIOS.

Thats the proxmox-boot-tool status output before rebooting:

root@pve-2:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
D557-E076 is configured with: grub (versions: 6.14.8-1-pve, 6.8.12-12-pve)
D558-30AB is configured with: grub (versions: 6.14.8-1-pve, 6.8.12-12-pve)

Thats the config von the nested pve-vm:

root@training4:~# qm config 202
agent: 0
bios: ovmf
boot: order=scsi0;scsi1
cores: 6
cpu: host
efidisk0: local-zfs:vm-202-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
ide2: local:iso/pve-b-autoinstall.iso,media=cdrom,size=1536128K
memory: 12000
meta: creation-qemu=9.0.0,ctime=1724752781
name: pve-2
net0: virtio=BC:24:11:7F:F9:09,bridge=pvea
net1: virtio=BC:24:11:BA:F4:3C,bridge=vmbr1,firewall=1
net2: virtio=BC:24:11:F8:B1:36,bridge=vmbr0,firewall=1
net3: virtio=BC:24:11:73:41:9A,bridge=vmbr0,firewall=1
net4: virtio=BC:24:11:6F:44:91,bridge=vmbr1,firewall=1
net5: virtio=BC:24:11:FA:CB:77,bridge=vmbr1,firewall=1
numa: 0
ostype: l26
parent: preupgrade
scsi0: local-zfs:vm-202-disk-1,discard=on,size=50G,ssd=1
scsi1: local-zfs:vm-202-disk-2,discard=on,size=50G,ssd=1
scsi2: local-zfs:vm-202-disk-3,discard=on,size=50G,ssd=1
scsi3: local-zfs:vm-202-disk-4,discard=on,size=50G,ssd=1
scsi4: local-zfs:vm-202-disk-5,discard=on,size=50G,ssd=1
scsi5: local-zfs:vm-202-disk-6,discard=on,size=50G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=c04c53e8-8428-493c-8d68-be72a27ddc32
sockets: 1
vmgenid: cde717ed-2356-46a1-8227-91b23ccdde29

I then checked proxmox-boot-tool. Did the following steps:

1. update-initramfs -u && proxmox-boot-tool refresh (reboot -> still broken)
2. proxmox-boot-tool init /dev/sda2 && proxmox-boot-tool init /dev/sdb2 && proxmox-boot-tool refresh (reboot -> pve9 is booting)

Is there something maybe missing in the upgrade-tutorial?

 

[t.lamprecht]

The nested installation is using ZFS, or?

After upgrading and rebooting, the VM does not boot anymore and goes to VM-BIOS.

Can you make it boot by selecting an entry there manually? Is there even an entry for proxmox in the boot loader menu?

 

[jsterr]

The nested installation is using ZFS, or?

Can you make it boot by selecting an entry there manually? Is there even an entry for proxmox in the boot loader menu?

Yes those vms are using 2 disks in a zfs mirror. Manually booting does not work it just goes back to bios again. This is how it looks like. I also only see one proxmox-entry.

 

[t.lamprecht]

Yes those vms are using 2 disks in a zfs mirror. Manually booting does not work it just goes back to bios again. This is how it looks like. I also only see one proxmox-entry.

View attachment 88217

And selecting that did not make it work?

Will try to reproduce, while I tried some nested upgrades, I'm not sure if it included a ZFS mirror one.

 

[jsterr]

And selecting that did not make it work?

Will try to reproduce, while I tried some nested upgrades, I'm not sure if it included a ZFS mirror one.

No unfortunately not. Thanks! On my side it only worked when doing the proxmox-boot-tool init /dev/sda2 && proxmox-boot-tool init /dev/sdb2 && proxmox-boot-tool refresh before rebooting.

 

[t.lamprecht]

Yeah, I could reproduce it in one setup, we'll take a closer look at this.