KVM internal-error on PVE 8.1 and Win7 guest.

[Budarov]

I have a cluster of AMD and PVE 7.4 processors.
I added 2 physical servers to the cluster with Xeon Gold 5412U and PVE 8.1

After migrating some virtual machines to a new server, I encountered a problem that after some time the guest crashed with an internal error.

I installed the latest updates. I tried changing the processor of the virtual machine to the "host", "kvm64", "x86-64-v2-AES".

Switched "Maсhine" to version "8.1".

I also tried to create a new virtual machine and reinstalled Win7 and got the same problem.

I also tried to solve the problem in the following ways:

echo "options kvm-intel nested=N" > /etc/modprobe.d/kvm-intel.conf

"split_lock_detect=off" in /etc/kernel/cmdline
None of the above worked.

Just repeat the problem. I made 10 clones of the machine, turn them off and turn them on through "Bulk Actions". It usually does not take more than three attempts for at least one virtual machine to receive an error.

Spoiler: pveversion -v

proxmox-ve: 8.1.0 (running kernel: 6.5.11-8-pve)
pve-manager: 8.1.4 (running version: 8.1.4/ec5affc9e41f1d79)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.5: 6.5.11-8
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
proxmox-kernel-6.5.11-7-pve-signed: 6.5.11-7
proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4
ceph-fuse: 17.2.7-pve1
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.8
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.4-1
proxmox-backup-file-restore: 3.1.4-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.4
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-3
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.2.0
pve-qemu-kvm: 8.1.5-2
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1

Spoiler: qm config

agent: 1
balloon: 0
boot: order=scsi0;ide2;net0
cores: 4
cpu: host
ide2: none,media=cdrom
machine: pc-i440fx-8.1
memory: 5120
meta: creation-qemu=6.1.0,ctime=1648991035
name: Copy-of-VM-Clone148
net0: e1000=BC:24:11:63:71:EE,bridge=vmbr0
numa: 0
onboot: 1
ostype: win7
scsi0: local-zfs:vm-920-disk-0,backup=0,discard=on,size=51304M,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=2023eac4-1171-4ee3-a65f-2c58ccb1e8c7
sockets: 1
vmgenid: 2d186dfc-60cd-4a7f-af0c-b5eaccebf9da

Spoiler: syslog err

eb 20 15:34:09 err QEMU[52189]: KVM internal error. Suberror: 3 │
│Feb 20 15:34:09 err QEMU[52189]: extra data[0]: 0x000000008000002f │
│Feb 20 15:34:09 err QEMU[52189]: extra data[1]: 0x0000000000000020 │
│Feb 20 15:34:09 err QEMU[52189]: extra data[2]: 0x0000000000000f82 │
│Feb 20 15:34:09 err QEMU[52189]: extra data[3]: 0x0000000000000017 │
│Feb 20 15:34:09 err QEMU[52189]: RAX=0000000000000000 RBX=fffff8a00165d974 RCX=0000000040000070 RDX=0000000000000000 │
│Feb 20 15:34:09 err QEMU[52189]: RSI=fffff8a00165d960 RDI=0000000000000000 RBP=fffff880021db640 RSP=fffff880021db598 │
│Feb 20 15:34:09 err QEMU[52189]: R8 =0000000000000001 R9 =fffff8a001734a70 R10=0000000000000001 R11=fffff8a000c23060 │
│Feb 20 15:34:09 err QEMU[52189]: R12=0000000000000000 R13=fffff8a001734a70 R14=0000000000000000 R15=0000000000000001 │
│Feb 20 15:34:09 err QEMU[52189]: RIP=fffff80002b0a12c RFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 │
│Feb 20 15:34:09 err QEMU[52189]: ES =002b 0000000000000000 ffffffff 00c0f300 DPL=3 DS [-WA] │
│Feb 20 15:34:09 err QEMU[52189]: CS =0010 0000000000000000 00000000 00209b00 DPL=0 CS64 [-RA] │
│Feb 20 15:34:09 err QEMU[52189]: SS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS [-WA] │
│Feb 20 15:34:09 err QEMU[52189]: DS =002b 0000000000000000 ffffffff 00c0f300 DPL=3 DS [-WA] │
│Feb 20 15:34:09 err QEMU[52189]: FS =0053 00000000fffae000 0000bc00 0040f300 DPL=3 DS [-WA] │
│Feb 20 15:34:09 err QEMU[52189]: GS =002b fffff88002ea9000 ffffffff 00c0f300 DPL=3 DS [-WA] │
│Feb 20 15:34:09 err QEMU[52189]: LDT=0000 0000000000000000 ffffffff 00c00000 │
│Feb 20 15:34:09 err QEMU[52189]: TR =0040 fffff88002eadf40 00000067 00008b00 DPL=0 TSS64-busy │
│Feb 20 15:34:09 err QEMU[52189]: GDT= fffff88002eb4540 0000007f │
│Feb 20 15:34:09 err QEMU[52189]: IDT= fffff88002eb45c0 00000fff │
│Feb 20 15:34:09 err QEMU[52189]: CR0=80050031 CR2=00000000000eeff8 CR3=00000001281c6000 CR4=000406f8 │
│Feb 20 15:34:09 err QEMU[52189]: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 │
│Feb 20 15:34:09 err QEMU[52189]: DR6=00000000fffe07f0 DR7=0000000000000400 │
│Feb 20 15:34:09 err QEMU[52189]: EFER=0000000000000d01 │
└Feb 20 15:34:09 err QEMU[52189]: Code=25 a8 4b 00 00 b9 70 00 00 40 0f ba 32 00 72 06 33 c0 8b d0 <0f> 30 5a 58 59 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 cc cc cc cc cc cc 66 66 0f 1f

 

[fiona]

Hi,
do you have the latest BIOS updates installed and also the latest intel-microcode package? https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_firmware_cpu

 

[Budarov]

Hi,
do you have the latest BIOS updates installed and also the latest intel-microcode package? https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_firmware_cpu

I will try to update the BIOS and microcode according to the instructions. The microcode version has changed, but this has no effect. I got the same error.

It seems to me that I was able to achieve stable operation by disabling hypertrading in the BIOS. But this makes the processor 20% slower, according to my tests.