Regression pve-qemu-kvm 5.1 fail to boot VM - grub rescue

[msg7086]

Last working version 5.0.0-13

First non-working version 5.1.0-1

Steps:

1. Upgrade pve-qemu-kvm to 5.1
2. Start VM in a specific config

Expected:

VM boots up as it was.

Actual:

Grub cannot find main drive.
error: no such device: xxx.
error: unknown filesystem.
Only (hd0) is found.
System stays on grub rescue> and cannot boot.

VM Config:

agent: 1
balloon: 0
boot: c
bootdisk: scsi1
cores: 4
cpu: host
machine: q35
memory: 4096
name: redacted
net0: virtio=66:66:95:0F:D0:13,bridge=vmbr0
numa: 0
ostype: l26
scsi0: vg0:vm-114-disk-0,cache=writeback,discard=on,size=16G,ssd=1
scsi1: local:114/vm-114-disk-0.qcow2,size=16M
scsihw: virtio-scsi-pci
smbios1: uuid=redacted
sockets: 1
vmgenid: redacted

/dev/vg0/vm-114-disk-0: UUID="06ec4971-ffa1-440c-8ea7-21bff10433fd" TYPE="xfs"

/dev/sda: UUID="06ec4971-ffa1-440c-8ea7-21bff10433fd" TYPE="xfs"
/dev/sdb1: PARTUUID="221f3302-ef45-4a48-847d-d499e3c7c2c2"

Disk /dev/sda: 16 GiB, 17179869184 bytes, 33554432 sectors
Disk model: QEMU HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 16 MiB, 16777216 bytes, 32768 sectors
Disk model: QEMU HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 2F25A1C7-C1A8-C541-B86B-E7B66DE7FC6B

Device     Start   End Sectors Size Type
/dev/sdb1   2048 30719   28672  14M BIOS boot

Description:
VM has 2 SCSI drives. sda is root, 16GB. sdb is boot drive, 16MB. sdb1 is BIOS boot partition that stores grub first stage.

 

[Stefan_R]

You're probably running into bug #3011, discussed here. The resolution is discussed at #3010.

In essence, the SeaBIOS version used for QEMU 5.1 only initializes drives for booting that are specified in the bootorder. Since pve-manager 6.2-13 and qemu-server 6.2-17 you can specify multiple boot drives, where you need to set both of your drives as enabled, with the one containing GRUB positioned first. You can use the GUI, or use qm set <vmid> -boot order=scsi1,scsi0 from the CLI.