Yes, this was added in Proxmox VE 8.2 (more specifically, in
pve-manager >= 8.2.0). See the feature request [1] for more information.[1] https://bugzilla.proxmox.com/show_bug.cgi?id=4474
[1] https://bugzilla.proxmox.com/show_bug.cgi?id=4474
i am stepping in into this, with even starting a container, the same issue happens.
i just started a created a container, and started it, i got the same ugly message, i was refreshing the desired page waiting for the LXC to load with no luck, then i went back to proxmox and
TASK ERROR: can't lock file '/run/lock/lxc/pve-config-100.lock' - got timeout
i just started a created a container, and started it, i got the same ugly message, i was refreshing the desired page waiting for the LXC to load with no luck, then i went back to proxmox and
TASK ERROR: can't lock file '/run/lock/lxc/pve-config-100.lock' - got timeout
Hi,
likely another task is already holding the lock. Please check the
Task History of the container to see if there are any other running tasks and also check the output of fuser -vau /run/lock/lxc/pve-config-100.lock.Thanks for the reply, however there is no task, it is newly created container in a newly created environment. I am crazy when it comes to testing, I rebuilt everything from scratch.
BTW, this is applied to all the containersni created.
Using 8.2.2
Are you sure the creation task is finished and there is not already a second start task? The
fuser command should indicate what is holding the lock.i am sure and here is another example from another machine and here is the output, where i cannot shutdown the machine at all. it has ubuntu server 22.04
there is something wrong with Virtual Environment 8.2.4 or 8.2 in general,
Code:
USER PID ACCESS COMMANDYou are great, I am using Version 8.2.7 and the same problem still persists, I was able to restart my server after doing what you mentioned.
Hello,
Probably in the same boat. When i click "reboot" my VM goes into a loop of shutting down, waiting ~7 seconds and automatically turning itself back on, waiting ~5min34secs and shutting down again. It can only be stopped by stopping it (no pun intended). I haven't found an answer to why it's happening, can anyone help with that? And how does one get into this loop? I'm pretty sure there was a time where i could shutdown a VM normally. I had messed up some permissions on my VM and i thought it may have "migrated" to the proxmox server and by that, it had lost permission to shutdown the VM. It's quite a relief to see others with the same problem, but also quite a worry to see it's such an old problem.
The loop stops with the "stop" command, but that's not a viable solution right? I should be able to tell a VM to shutdown and then it does so gracefully.
I've seen Proxmox team members wanting outputs from following commands:
(More output of the journalctl command can be found at https://pastebin.com/MCNEj13y )
Probably in the same boat. When i click "reboot" my VM goes into a loop of shutting down, waiting ~7 seconds and automatically turning itself back on, waiting ~5min34secs and shutting down again. It can only be stopped by stopping it (no pun intended). I haven't found an answer to why it's happening, can anyone help with that? And how does one get into this loop? I'm pretty sure there was a time where i could shutdown a VM normally. I had messed up some permissions on my VM and i thought it may have "migrated" to the proxmox server and by that, it had lost permission to shutdown the VM. It's quite a relief to see others with the same problem, but also quite a worry to see it's such an old problem.
The loop stops with the "stop" command, but that's not a viable solution right? I should be able to tell a VM to shutdown and then it does so gracefully.
I've seen Proxmox team members wanting outputs from following commands:
(More output of the journalctl command can be found at https://pastebin.com/MCNEj13y )
Code:
root@pve:~# fuser -vau /run/lock/qemu-server/lock-112.conf
USER PID ACCESS COMMAND
/run/lock/qemu-server/lock-112.conf:
root@pve:~#
root@pve:~# pveversion -v
proxmox-ve: 8.2.0 (running kernel: 6.8.12-4-pve)
pve-manager: 8.2.8 (running version: 8.2.8/a577cfa684c7476d)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.12-4
proxmox-kernel-6.8.12-4-pve-signed: 6.8.12-4
proxmox-kernel-6.8.8-2-pve-signed: 6.8.8-2
proxmox-kernel-6.5.13-6-pve-signed: 6.5.13-6
proxmox-kernel-6.5: 6.5.13-6
proxmox-kernel-6.5.13-5-pve-signed: 6.5.13-5
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
ceph-fuse: 17.2.7-pve3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx9
intel-microcode: 3.20241112.1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.4
libpve-access-control: 8.1.4
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.8
libpve-cluster-perl: 8.0.8
libpve-common-perl: 8.2.8
libpve-guest-common-perl: 5.1.4
libpve-http-server-perl: 5.1.2
libpve-network-perl: 0.9.8
libpve-rs-perl: 0.8.11
libpve-storage-perl: 8.2.6
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.5.0-1
proxmox-backup-client: 3.2.9-1
proxmox-backup-file-restore: 3.2.9-1
proxmox-firewall: 0.5.0
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.0
pve-cluster: 8.0.8
pve-container: 5.2.1
pve-docs: 8.2.4
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.2
pve-firewall: 5.0.7
pve-firmware: 3.14-1
pve-ha-manager: 4.0.5
pve-i18n: 3.2.4
pve-qemu-kvm: 9.0.2-4
pve-xtermjs: 5.3.0-3
qemu-server: 8.2.6
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.6-pve1
root@pve:~#
root@pve:~# lsof /var/lock/qemu-server/lock-112.conf
root@pve:~#
root@pve:~# qm config 112
agent: 1
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 6
cpu: x86-64-v3
efidisk0: lvm-data:vm-112-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
ide2: z-iso:iso/ubuntu-24.04.1-live-server-amd64.iso,media=cdrom,size=2708862K
machine: q35
memory: 4096
meta: creation-qemu=9.0.2,ctime=1732446950
name: docker-media-stack2
net0: virtio=BC:24:11:FE:72:AC,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: lvm-data:vm-112-disk-1,iothread=1,size=80G
scsihw: virtio-scsi-single
smbios1: uuid=74c417d1-9a63-4728-b24b-d98d487a0fce
sockets: 1
vcpus: 6
vmgenid: 6d2e1b77-dda5-4ca0-a398-1444eb4dc1bf
root@pve:~#
(Newly created and installed ubuntu VM)
root@pve:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
192.168.1.19 pve.mgmt pve
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
root@pve:~#
root@pve:~# hostname
pve
root@pve:~#
Started new reboot process for VM 112 at 13:38:05
journalctl:
Nov 24 13:38:05 pve pvedaemon[1766995]: <root@pam> starting task UPID:pve:001E6092:01959AD0:67431E2D:qmreboot:112:root@pam:
Nov 24 13:38:05 pve pvedaemon[1990802]: requesting reboot of VM 112: UPID:pve:001E6092:01959AD0:67431E2D:qmreboot:112:root@pam:
Nov 24 13:38:17 pve pvedaemon[1786795]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - got timeout
Nov 24 13:38:36 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - got timeout
Nov 24 13:38:55 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:39:14 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:39:37 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:40:02 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:40:25 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:40:38 pve kernel: eth0: entered promiscuous mode
Nov 24 13:40:47 pve qm[1992670]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:40:49 pve kernel: eth0: left promiscuous mode
Nov 24 13:40:49 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:40:51 pve qm[1992829]: <root@pam> starting task UPID:pve:001E687E:0195DBBB:67431ED3:qmstop:112:root@pam:
Nov 24 13:40:51 pve qm[1992830]: stop VM 112: UPID:pve:001E687E:0195DBBB:67431ED3:qmstop:112:root@pam:
Nov 24 13:41:01 pve qm[1992830]: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:41:01 pve qm[1992829]: <root@pam> end task UPID:pve:001E687E:0195DBBB:67431ED3:qmstop:112:root@pam: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:41:06 pve qm[1992996]: start VM 112: UPID:pve:001E6924:0195E1CF:67431EE2:qmstart:112:root@pam:
Nov 24 13:41:06 pve qm[1992995]: <root@pam> starting task UPID:pve:001E6924:0195E1CF:67431EE2:qmstart:112:root@pam:
Nov 24 13:41:13 pve pvedaemon[1786795]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:41:16 pve qm[1992996]: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:41:16 pve qm[1992995]: <root@pam> end task UPID:pve:001E6924:0195E1CF:67431EE2:qmstart:112:root@pam: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:41:37 pve pvedaemon[1786795]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:42:02 pve pvedaemon[1786795]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:42:25 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:42:47 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:43:11 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:43:35 pve pvedaemon[1786795]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:43:57 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:44:20 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:44:43 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:45:06 pve pvedaemon[1766995]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:45:29 pve pvedaemon[1947822]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:45:51 pve kernel: eth0: entered promiscuous mode
Nov 24 13:46:02 pve kernel: eth0: left promiscuous mode
Nov 24 13:46:47 pve qm[1996794]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - unable to connect to VM 112 qga socket - timeout after 31 retries
Nov 24 13:46:50 pve qm[1996861]: <root@pam> starting task UPID:pve:001E783E:01966833:6743203A:qmstop:112:root@pam:
Nov 24 13:46:50 pve qm[1996862]: stop VM 112: UPID:pve:001E783E:01966833:6743203A:qmstop:112:root@pam:
Nov 24 13:47:00 pve qm[1996862]: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:47:00 pve qm[1996861]: <root@pam> end task UPID:pve:001E783E:01966833:6743203A:qmstop:112:root@pam: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:47:06 pve qm[1997041]: <root@pam> starting task UPID:pve:001E78F2:01966E44:6743204A:qmstart:112:root@pam:
Nov 24 13:47:06 pve qm[1997042]: start VM 112: UPID:pve:001E78F2:01966E44:6743204A:qmstart:112:root@pam:
Nov 24 13:47:16 pve qm[1997042]: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:47:16 pve qm[1997041]: <root@pam> end task UPID:pve:001E78F2:01966E44:6743204A:qmstart:112:root@pam: can't lock file '/var/lock/qemu-server/lock-112.conf' - got timeout
Nov 24 13:48:05 pve pvedaemon[1990802]: VM 112 qmp command failed - VM 112 qmp command 'guest-shutdown' failed - got timeout
Nov 24 13:48:05 pve pvedaemon[1990802]: VM quit/powerdown failed
Nov 24 13:48:05 pve pvedaemon[1766995]: <root@pam> end task UPID:pve:001E6092:01959AD0:67431E2D:qmreboot:112:root@pam: VM quit/powerdown failed
Nov 24 13:49:05 pve pvedaemon[1947822]: <root@pam> successful auth for user 'root@pam'
Nov 24 13:49:26 pve smartd[1324]: Device: /dev/sdb [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 103 to 104
Nov 24 13:49:26 pve smartd[1324]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 102 to 103
Nov 24 13:49:26 pve smartd[1324]: Device: /dev/sdd [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 101 to 102
Nov 24 13:49:26 pve smartd[1324]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 103 to 104
Nov 24 13:49:26 pve smartd[1324]: Device: /dev/sdf [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 105 to 106
Nov 24 13:50:01 pve kernel: eth0: entered promiscuous mode
Nov 24 13:50:12 pve kernel: eth0: left promiscuous mode
Nov 24 13:52:47 pve qm[2000812]: VM 112 qmp command failed - VM 112 qmp command 'guest-ping' failed - got timeout
Nov 24 13:52:50 pve qm[2000900]: stop VM 112: UPID:pve:001E8804:0196F497:674321A2:qmstop:112:root@pam:
Nov 24 13:52:50 pve qm[2000895]: <root@pam> starting task UPID:pve:001E8804:0196F497:674321A2:qmstop:112:root@pam:
Nov 24 13:52:50 pve kernel: tap112i0: left allmulticast mode
Nov 24 13:52:50 pve kernel: fwbr112i0: port 2(tap112i0) entered disabled state
Nov 24 13:52:50 pve kernel: fwbr112i0: port 1(fwln112i0) entered disabled state
Nov 24 13:52:50 pve kernel: vmbr0: port 5(fwpr112p0) entered disabled state
Nov 24 13:52:50 pve kernel: fwln112i0 (unregistering): left allmulticast mode
Nov 24 13:52:50 pve kernel: fwln112i0 (unregistering): left promiscuous mode
Nov 24 13:52:50 pve kernel: fwbr112i0: port 1(fwln112i0) entered disabled state
Nov 24 13:52:50 pve kernel: fwpr112p0 (unregistering): left allmulticast mode
Nov 24 13:52:50 pve kernel: fwpr112p0 (unregistering): left promiscuous mode
Nov 24 13:52:50 pve kernel: vmbr0: port 5(fwpr112p0) entered disabled state
Nov 24 13:52:50 pve qmeventd[1325]: read: Connection reset by peer
Nov 24 13:52:50 pve qm[2000895]: <root@pam> end task UPID:pve:001E8804:0196F497:674321A2:qmstop:112:root@pam: OK
Nov 24 13:52:50 pve systemd[1]: 112.scope: Deactivated successfully.
Nov 24 13:52:50 pve systemd[1]: 112.scope: Consumed 1min 38.710s CPU time.
Nov 24 13:52:51 pve qmeventd[2000921]: Starting cleanup for 112
Nov 24 13:52:51 pve qmeventd[2000921]: Finished cleanup for 112
Nov 24 13:52:56 pve qm[2000993]: <root@pam> starting task UPID:pve:001E8862:0196F6DF:674321A8:qmstart:112:root@pam:
Nov 24 13:52:56 pve qm[2000994]: start VM 112: UPID:pve:001E8862:0196F6DF:674321A8:qmstart:112:root@pam:
Nov 24 13:52:56 pve systemd[1]: Started 112.scope.
Nov 24 13:52:56 pve kernel: tap112i0: entered promiscuous mode
Nov 24 13:52:56 pve kernel: vmbr0: port 5(fwpr112p0) entered blocking state
Nov 24 13:52:56 pve kernel: vmbr0: port 5(fwpr112p0) entered disabled state
Nov 24 13:52:56 pve kernel: fwpr112p0: entered allmulticast mode
Nov 24 13:52:56 pve kernel: fwpr112p0: entered promiscuous mode
Nov 24 13:52:56 pve kernel: vmbr0: port 5(fwpr112p0) entered blocking state
Nov 24 13:52:56 pve kernel: vmbr0: port 5(fwpr112p0) entered forwarding statealmost all such issues come from a certain monitoring script, see: https://forum.proxmox.com/threads/vm-resetting-after-every-6-minutes.127110/post-556621