Display Type VirGL GPU (virtio-gl) Issues w/ Suspend & Snapshots

C

crfslick

New Member
Nov 23, 2023
3
0
1
Linux PVE guests with display types set to "virtio-gl" don't resume properly from hibernation and snapshots. Resuming results in no display output and requires resetting the guest to fix.

The the only sign that something has gone wrong is in the task output (no errors in dmesg).

Code: 
Resuming suspended VM
activating and using 'local-storage:vm-212-state-suspend-2023-11-23' as vmstate
kvm: Failed to load virtio-gpu:virtio-gpu
kvm: error while loading state for instance 0x0 of device '0000:00:01.0/virtio-gpu'
kvm: Error while loading VM state: Invalid argument
Resumed VM, removing state
TASK OK

For context, the PVE host is running an AMD Ryzen 5000 series "G" skew cpu. Issue is present with both SeaBIOS and OVMF machine configurations. Required libraries "libgl1" and "libegl1" are installed, as per the documentation, and virtio-gl generally works fine otherwise.

Is anyone else experiencing this issue?

Edit: host is running PVE 8.1.3, kernel 6.5.11-4-pve
 
Last edited:
Yes, sounds reasonable. You're breaking the strict virtualization by using hardware (virtio-gpu), which has not yet a snapshotable or hibernatable state. Maybe it'll be in the future, but this is far from trivial and may lead (like you discovered) to some problems. Easiest way is to snapshot in offline mode and don't hibernate - like with everything else. Online snapshot is always risky.
 
LnxBil said:
Yes, sounds reasonable. You're breaking the strict virtualization by using hardware (virtio-gpu), which has not yet a snapshotable or hibernatable state. Maybe it'll be in the future, but this is far from trivial and may lead (like you discovered) to some problems. Easiest way is to snapshot in offline mode and don't hibernate - like with everything else. Online snapshot is always risky.
Click to expand...
Makes sense. I guess I didn't realize how close virtio-gpu was to actual hardware pass-through. Thanks

Although it would be nice if the GUI informed the user of this when hibernating / making a snapshot of a system with virtio-gpu enabled. As it is, the resume / restore task "succeeds" (TASK OK) but fails silently with errors that are only found in the task log which isn't shown by default.
 
crfslick said:
Although it would be nice if the GUI informed the user of this when hibernating / making a snapshot of a system with virtio-gpu enabled. As it is, the resume / restore task "succeeds" (TASK OK) but fails silently with errors that are only found in the task log which isn't shown by default.
Click to expand...
Agreed.
 
Hi,
can you please share the configuration of an affected guest qm config <ID> and output of pveversion -v. Usually QEMU complains up front about migration blockers (snapshots/hibernation use the same logic/code under the hood), it's a bit strange it doesn't consider this to be one.
 
fiona said:
Hi,
can you please share the configuration of an affected guest qm config <ID> and output of pveversion -v. Usually QEMU complains up front about migration blockers (snapshots/hibernation use the same logic/code under the hood), it's a bit strange it doesn't consider this to be one.
Click to expand...
Code: 
agent: 1
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 4
cpu: x86-64-v2-AES
efidisk0: local-storage:vm-212-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
ide2: remote-storage:iso/ubuntu-20.04.3-desktop-amd64.iso,media=cdrom,size=2999936K
machine: q35
memory: 8192
meta: creation-qemu=8.0.2,ctime=1700759230
name: test
net0: virtio=22:75:83:8D:F7:82,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-storage:vm-212-disk-1,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=17d017d9-1dd2-4e2a-adf0-a302e56e6d78
sockets: 1
vga: virtio-gl
vmgenid: b33af661-f907-44d8-918e-f498728accba

Code: 
proxmox-ve: 8.1.0 (running kernel: 6.5.11-4-pve)
pve-manager: 8.1.3 (running version: 8.1.3/b46aac3b42da5d15)
proxmox-kernel-helper: 8.0.9
pve-kernel-6.2: 8.0.5
proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4
proxmox-kernel-6.5: 6.5.11-4
proxmox-kernel-6.2.16-19-pve: 6.2.16-19
proxmox-kernel-6.2: 6.2.16-19
proxmox-kernel-6.2.16-15-pve: 6.2.16-15
ceph-fuse: 16.2.11+ds-2
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx7
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.0
libproxmox-rs-perl: 0.3.1
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.4
libpve-rs-perl: 0.8.7
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve3
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.0.4-1
proxmox-backup-file-restore: 3.0.4-1
proxmox-kernel-helper: 8.0.9
proxmox-mail-forward: 0.2.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.2
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-1
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.1.2
pve-qemu-kvm: 8.1.2-4
pve-xtermjs: 5.3.0-2
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.0-pve3
 
Just checking-in on this issue since it appears to continue being an issue with VMs that use VirGL for a Display. The result of hibernation results in the following output, and the VM is stuck in a bad state and must be unlocked before it can be powered on again (with the loss of the hibernation state).

Task viewer: VM 101 - Hibernate

Formatting '/var/lib/vz/images/101/vm-101-state-suspend-2025-06-08.raw', fmt=raw size=21999124480 preallocation=off
VM 101 not running

TASK ERROR: VM 101 qmp command 'query-savevm' failed - client closed connection

This is on Proxmox 8.4.1 with all of the latest non-subscription updates as of today (June 8, 2025).
 
Hi,
nightowl said:
Just checking-in on this issue since it appears to continue being an issue with VMs that use VirGL for a Display. The result of hibernation results in the following output, and the VM is stuck in a bad state and must be unlocked before it can be powered on again (with the loss of the hibernation state).

Task viewer: VM 101 - Hibernate

Formatting '/var/lib/vz/images/101/vm-101-state-suspend-2025-06-08.raw', fmt=raw size=21999124480 preallocation=off
VM 101 not running

TASK ERROR: VM 101 qmp command 'query-savevm' failed - client closed connection

This is on Proxmox 8.4.1 with all of the latest non-subscription updates as of today (June 8, 2025).
Click to expand...
yours is a crash during saving the VM state, not an error during loading like the original issue in this thread.

To obtain a backtrace (which would be very helpful for debugging), run apt install pve-qemu-kvm-dbgsym gdb systemd-coredump. After triggering the issue again (on a test VM, e.g. a clone), you can run coredumpctl -1 gdb and then in the GDB prompt thread apply all backtrace. Please also share the config of that VM, i.e. qm config <ID>.
 
Last edited:
You must log in or register to reply here.
Top Bottom