VM hanging / freeze after update from PVE 6 to 7

[shruxx88]

I cannot start a simple rescue disc in a new vm either.
EDIT: it is possible, but only if I boot in UEFI mode. My VMs running in SeaBIOS, so this would not be an option.
EDIT2: UEFI is not working in every scenario - only a fresh configured vm without anything will boot a rescue disc, nothing else...

 

[shruxx88]

So after a long while... I finally made the VMs boot. What did I do? I've booted the pve server with an older kernel - that fixed the issue for now. Hopefully I'll remember this workaround ;D
I've checked every logfile for errors with the newest kernel and there was NOTHING... but thanks for the help!

 

[mira]

There's something in the log:

Nov 18 20:06:12 proxmox kernel: unchecked MSR access error: WRMSR to 0xda0 (tried to write 0x0000000000000000) at rIP: 0xffffffff8c07fb24 (native_write_msr+0x4/0x30)
Nov 18 20:06:12 proxmox kernel: Call Trace:
Nov 18 20:06:12 proxmox kernel:  ? fpu__init_cpu_xstate+0xb3/0x100
Nov 18 20:06:12 proxmox kernel:  fpu__init_system_xstate+0x147/0x9b9
Nov 18 20:06:12 proxmox kernel:  fpu__init_system+0x118/0x151
Nov 18 20:06:12 proxmox kernel:  early_cpu_init+0x410/0x445
Nov 18 20:06:12 proxmox kernel:  setup_arch+0x5c/0xc16
Nov 18 20:06:12 proxmox kernel:  ? printk+0x58/0x6f
Nov 18 20:06:12 proxmox kernel:  start_kernel+0x6f/0x85c
Nov 18 20:06:12 proxmox kernel:  x86_64_start_reservations+0x24/0x26
Nov 18 20:06:12 proxmox kernel:  x86_64_start_kernel+0x8b/0x8f

According to the log this is a VPS from netcup? They probably run an old kernel/hypervisor that's not yet compatible with kernel 5.13.

 

[shruxx88]

There's something in the log:

Nov 18 20:06:12 proxmox kernel: unchecked MSR access error: WRMSR to 0xda0 (tried to write 0x0000000000000000) at rIP: 0xffffffff8c07fb24 (native_write_msr+0x4/0x30)
Nov 18 20:06:12 proxmox kernel: Call Trace:
Nov 18 20:06:12 proxmox kernel:  ? fpu__init_cpu_xstate+0xb3/0x100
Nov 18 20:06:12 proxmox kernel:  fpu__init_system_xstate+0x147/0x9b9
Nov 18 20:06:12 proxmox kernel:  fpu__init_system+0x118/0x151
Nov 18 20:06:12 proxmox kernel:  early_cpu_init+0x410/0x445
Nov 18 20:06:12 proxmox kernel:  setup_arch+0x5c/0xc16
Nov 18 20:06:12 proxmox kernel:  ? printk+0x58/0x6f
Nov 18 20:06:12 proxmox kernel:  start_kernel+0x6f/0x85c
Nov 18 20:06:12 proxmox kernel:  x86_64_start_reservations+0x24/0x26
Nov 18 20:06:12 proxmox kernel:  x86_64_start_kernel+0x8b/0x8f

According to the log this is a VPS from netcup? They probably run an old kernel/hypervisor that's not yet compatible with kernel 5.13.

Yes, it is.
It is pretty stable right now with the older kernel. Will boot the hv with new kernel if netcup updates his host kernel.

 

[pschonmann]

Hi same problem here

imgstorage is vps (nfs server) in pve7 ( node1-ryzen )
-

proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve)
pve-manager: 7.1-8 (running version: 7.1-8/5b267f33)
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

-

Jan 19 15:30:12 node1-ryzen kernel: [2922747.719152] clocksource: timekeeping watchdog on CPU29: hpet read-back delay of 168946ns, attempt 4, marking unstable
Jan 19 15:30:12 node1-ryzen kernel: [2922747.719328] tsc: Marking TSC unstable due to clocksource watchdog
Jan 19 15:30:12 node1-ryzen kernel: [2922747.719335] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
Jan 19 15:30:12 node1-ryzen kernel: [2922747.719335] sched_clock: Marking unstable (2922751350090211, -3630754782)<-(2922747794514101, -75179992)
Jan 19 15:30:12 node1-ryzen kernel: [2922747.721128] clocksource: Checking clocksource tsc synchronization from CPU 29.
Jan 19 15:30:12 node1-ryzen kernel: [2922747.725084] clocksource: Switched to clocksource hpet
Jan 19 15:34:01 node1-ryzen kernel: [2922976.998652] nfs: server imgstorage not responding, timed out
Jan 19 15:37:02 node1-ryzen kernel: [2923157.218673] nfs: server imgstorage not responding, timed out
Jan 19 15:37:07 node1-ryzen kernel: [2923162.338661] nfs: server imgstorage not responding, timed out
Jan 19 15:40:02 node1-ryzen kernel: [2923337.442672] nfs: server imgstorage not responding, timed out
Jan 19 15:40:12 node1-ryzen kernel: [2923347.682651] nfs: server imgstorage not responding, timed out
Jan 19 15:41:11 node1-ryzen kernel: [2923406.487120] vmbr0: port 14(tap101i0) entered disabled state
Jan 19 15:41:11 node1-ryzen kernel: [2923406.713788] vmbr27: port 6(tap101i1) entered disabled state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.243211] device tap101i0 entered promiscuous mode
Jan 19 15:41:15 node1-ryzen kernel: [2923410.250822] vmbr0: port 14(tap101i0) entered blocking state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.250825] vmbr0: port 14(tap101i0) entered disabled state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.251222] vmbr0: port 14(tap101i0) entered blocking state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.251223] vmbr0: port 14(tap101i0) entered forwarding state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.571040] device tap101i1 entered promiscuous mode
Jan 19 15:41:15 node1-ryzen kernel: [2923410.579758] vmbr27: port 6(tap101i1) entered blocking state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.579760] vmbr27: port 6(tap101i1) entered disabled state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.579833] vmbr27: port 6(tap101i1) entered blocking state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.579834] vmbr27: port 6(tap101i1) entered forwarding state
Jan 19 15:41:15 node1-ryzen kernel: [2923410.605216] kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable
Jan 19 15:43:02 node1-ryzen kernel: [2923517.666680] nfs: server imgstorage not responding, timed out
Jan 19 15:43:17 node1-ryzen kernel: [2923533.026683] nfs: server imgstorage not responding, timed out
Jan 19 15:46:02 node1-ryzen kernel: [2923697.890685] nfs: server imgstorage not responding, timed out
Jan 19 15:46:23 node1-ryzen kernel: [2923718.370687] nfs: server imgstorage not responding, timed out
Jan 19 15:49:03 node1-ryzen kernel: [2923878.114675] nfs: server imgstorage not responding, timed out
Jan 19 15:49:29 node1-ryzen kernel: [2923904.738683] nfs: server imgstorage not responding, timed out
Jan 19 15:52:03 node1-ryzen kernel: [2924058.342678] nfs: server imgstorage not responding, timed out
Jan 19 15:52:03 node1-ryzen kernel: [2924058.554121] vmbr0: port 7(tap108i0) entered disabled state
Jan 19 15:52:13 node1-ryzen kernel: [2924068.770690] device tap108i0 entered promiscuous mode
Jan 19 15:52:13 node1-ryzen kernel: [2924068.780089] vmbr0: port 7(tap108i0) entered blocking state
Jan 19 15:52:13 node1-ryzen kernel: [2924068.780092] vmbr0: port 7(tap108i0) entered disabled state
Jan 19 15:52:13 node1-ryzen kernel: [2924068.780484] vmbr0: port 7(tap108i0) entered blocking state
Jan 19 15:52:13 node1-ryzen kernel: [2924068.780485] vmbr0: port 7(tap108i0) entered forwarding state
Jan 19 15:52:36 node1-ryzen kernel: [2924091.106690] nfs: server imgstorage not responding, timed out

When problem occurs, all debian wheezy (deb7) vms on that hypervisor stucked. Rebooted then qm reset XXX and works fine now.