VMs will not shutdown clean anymore

[Shadow Sysop]

I'm not sure exactly when this started happening, but VMs no longer shutdown in a clean way, whether initiated from inside the VM or via Shutdown signal from Proxmox GUI. On a shutdown, the VM will hang, requiring a Stop command to be issued in order to shutdown the machine fully. Sometimes, even the stop command does not work, requiring a kill of the VM process from the the host. This is consistent across different OS on each VM (Windows, Ubuntu, CentOS).

pve-manager/7.3-4/d69b70d4 (running kernel: 5.15.39-2-pve)

 

[Neobin]

pve-manager/7.3-4/d69b70d4 (running kernel: 5.15.39-2-pve)

This kernel is around half a year old. (Sidenote: PVE 7.3 was released with kernel: 5.15.74.)
Not going to say for sure, that this is your problem, but to start with, I would make sure that the PVE-host is up-to-date and then give it a reboot to boot with a recent kernel. (Or did you pin that kernel on purpose? If yes, why?)

In addition:

• Standalone node or cluster?
  • If cluster, how does it look like, if you migrate a misbehaving VM to another node and shut it down there?
  • Are all nodes in the cluster on the same PVE-version and...
  • ...also running the same (old) kernel-version?
• Qemu-guest-agent installed?
  • If yes, is it also enabled in the VM-options?
  • Or disabled, if it is not installed! (Important!)
• Please provide the full VM-config (qm config VMID) in code-tags of at least one misbehaving VM.
• Anything in the syslog of the PVE-host and/or the guests?
• Does this happen with all VMs? Not a single one, that shuts down properly?
  • Everytime or now and then?
• LXCs behave normal?

 

[Shadow Sysop]

• Standalone node or cluster?
  • Cluster, but other in cluster are different hardware so this may or may not be a viable test.
  • Yes
  • No. This node is only rebooted once or twice per year.
• Qemu-guest-agent installed?
  • yes and yes
• Please provide the full VM-config (qm config VMID) in code-tags of at least one misbehaving VM.

root@server9:~# qm config 122
agent: 1,fstrim_cloned_disks=1
balloon: 1024
boot: c
bootdisk: scsi0
cipassword: **********
ciuser: root
cores: 4
cpu: host
ide2: local:122/vm-122-cloudinit.qcow2,media=cdrom,size=4M
ipconfig0: ip=12.345.678.9/32,gw=12.34.56.789
memory: 4096
name: 8-openvpn
net0: virtio=02:00:00:2b:ed:1d,bridge=vmbr0
numa: 1
onboot: 1
ostype: l26
scsi0: local:122/vm-122-disk-0.raw,cache=writeback,size=25804M
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=97da9897-6862-47b1-8db0-df097761094f
sockets: 1
vmgenid: d9251d09-669f-4b23-a490-91e195ab09e8

• Anything in the syslog of the PVE-host and/or the guests?
  Nothing of note
• 
  
• Does this happen with allVMs? Not a single one, that shuts down properly?
  • All VMs. Never do they Shutdown properly. But they'll Stop properly if Shutdown wasn't initiated first.
• LXCs behave normal?

We don't use LXC

Thank you for your assistance in this matter. The node is in production and cannot really be rebooted except in critical circumstance.

 

[Shadow Sysop]

I have resolved the issue by running the following commands on the host

systemctl daemon-reload and then systemctl start qmeventd.service