VM is locked (rollback) and Could not open '/dev/pve/vm-100-disk-0'

[naupe]

I have an Nginx VM that will not start. It was working yesterday. When I try to start it, I receive:

VM is locked (rollback)

When I unlock the VM with qm unlock 100, and try to start it with qm start 100, I receive the following:

kvm: -drive file=/dev/pve/vm-100-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on: Could not open '/dev/pve/vm-100-disk-0': No such file or directory
start failed: command '/usr/bin/kvm -id 100 -name Nginx -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=d428ca8e-75e8-46e3-ae2e-b750a532dc95' -smp '6,sockets=2,cores=3,maxcpus=6' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 6144 -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'vmgenid,guid=a19adad0-ee14-4c3a-aa8b-cf5f913971fd' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'VGA,id=vga,bus=pci.0,addr=0x2' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:74fa627f1f2' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/pve/vm-100-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=0E:7E:43:01:2F:7A,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -machine 'type=pc'' failed: exit code 1

I can't rollback the VM either:

lvremove 'pve/vm-100-disk-0' error:   Failed to find logical volume "pve/vm-100-disk-0"

Any idea what is going on!? This is the first time this has ever happened before.

------------------------------------


EDIT: Upon further review, I've found that there is no vm-100-disk-0 file within /dev/pve. I have no idea how this would have happened, as I am the only one with access to this Server. Is there some way I can create vm-100-disk-0 and then rollback to one of the working Snapshots of this VM? I still have the Snapshots in the VM. Just one idea of mine right now.

 

[naupe]

I have great news: someone on Reddit helped me resolve the issue!

Ran the following in Proxmox to re-create the Volume Group:

lvcreate -L11G -n vm-100-disk-0 pve

(Nginx VM is 11G in disk size and pve is the Volume Group)

Nginx now had a Hard Drive, and I was able to restore the Nginx VM to a point where it was working, and now all my sites that Reverse Proxy through Nginx are working!

EDIT: I said before that Certbot in Nginx was throwing some strange errors, but they went away after a few minutes when I renewed the certs. I suspect strongly that it was related to the de-synced time on the Nginx VM (as will happen immediately after a Snapshot Rollback).

I now would like to figure out WHY vm-100-disk-0 was deleted in the first place. Looking through the history on Proxmox, I see nothing that suggests it was a manual deletion, and I see no suspicious logins. I can only assume it was an automatic deletion by Proxmox itself. Is local-lvm set to automatically delete a Disk Drive if it fills up? I will say that my local-lvm is at 95% capacity right now, and I'm doing what I can to lower it while keeping the really important Snapshots, but that would certainly be an odd behavior of local-lvm to delete a whole Virtual Disk, right?

 

[theanimalbae]

qm unlock 100 && rm -rf /var/lock/qemu-server/lock-*

might want to copy over your back up version VMdrive. You either have corrupted VMdrive or are impatient and stopped the rolling back a snapshot half way.