[SOLVED] VMs not working anymore after PVE 7 upgrade

[fabian]

okay. could you collect the output of lsof /dev/mapper/vm-106-disk-0? thanks!

 

[fabian]

is your friend also using (local) LVM as storage backend?

 

[Spinning_rust]

okay. could you collect the output of lsof /dev/mapper/vm-106-disk-0? thanks!

Yep, i had to change your command slightly but i got the vmdisk 106:

root@top:~# lsof /dev/mapper/PLS-vm--106--disk--0
COMMAND    PID USER   FD   TYPE DEVICE  SIZE/OFF NODE NAME
fdisk   281530 root    3u   BLK  253,8 0t2098176  604 /dev/mapper/../dm-8

 

[Spinning_rust]

is your friend also using (local) LVM as storage backend?

No, he was using old laptops and single disk zfs pool as a storage backend for VMs and containers

 

[Rocha Neto]

I just realized that. Reboot cannot be done after upgrade.

I will perform as soon as possible.

After rebooting, everything works fine. Backup with PBS, VMs, everything. Thanks.

 

[fabian]

Yep, i had to change your command slightly but i got the vmdisk 106:

root@top:~# lsof /dev/mapper/PLS-vm--106--disk--0
COMMAND    PID USER   FD   TYPE DEVICE  SIZE/OFF NODE NAME
fdisk   281530 root    3u   BLK  253,8 0t2098176  604 /dev/mapper/../dm-8

I am not sure what else there is we can check (I assume this is the fdisk process you used to look at the partition table?).. if you have an older backup you could extract the partition layout from there and re-create it

 

[Spinning_rust]

I am not sure what else there is we can check (I assume this is the fdisk process you used to look at the partition table?).. if you have an older backup you could extract the partition layout from there and re-create it

Yeah I think I have an older backup, I could try that tomorrow when I’m back in the Office

about the command, no i just changed the filename (?) to vmdisk 106-0

But it‘s still weird, it‘s like the new version of qemu isn’t compatible with the „old“ disk format or something

 

[fabian]

Yeah I think I have an older backup, I could try that tomorrow when I’m back in the Office

about the command, no i just changed the filename (?) to vmdisk 106-0

But it‘s still weird, it‘s like the new version of qemu isn’t compatible with the „old“ disk format or something

no, qemu just sees what is there - a basically 'blank' disk/volume. qemu also does not overwrite the disk on its own unless the guest OS tells it to. the most likely error source would be the storage itself (in this case either the iSCSI backend, or LVM)..

 

[Spinning_rust]

no, qemu just sees what is there - a basically 'blank' disk/volume. qemu also does not overwrite the disk on its own unless the guest OS tells it to. the most likely error source would be the storage itself (in this case either the iSCSI backend, or LVM)..

i restored some VMs with older backups and that "solved" the issue, as theyre now booting up again.

I'm still curious how exactly the upgrade (or whatever else happened at this time) broke the VM disks because the VMs were 100% working before i powered them off for the upgrade. Hopefully it doesnt happen to anyone else

I'm marking this thread as solved now

 

[RolandK]

have you been able to restore the partition tables with testdisk or gpart or similar tool ?

there is a bugreport at https://bugzilla.proxmox.com/show_bug.cgi?id=2874 regrading loss of partition table information possibly related to vzdump, but your issue looks different. not sure if it's related, though.