Nested PVE (on PVE host) Kernel panic Host injected async #PF in kernel mode

Seeing this issue with nested PVE on PVE on EPYC 9124 CPU. Host has SWAP and KSM enabled, although it has plenty of free memory (100GB of 512GB). This cluster currently runs PVE8.4.14 and runs other workloads too besides nested PVE VMs, with Debian, Ubuntu and Windows Server guest OS. Only the nested PVE VMs are affected by the kernel panic mentioned in this thread.
 
I will update both host and guest to latest pve kernel and test when a window becomes available and will report here after.
Let's hope it was fixed at some point...
 
Sadly it failed again...

Again another Async Page Fault

Host: pve-manager/9.2.3/d0fde103346cf89a (running kernel: 7.0.6-2-pve)
Guest: pve-manager/9.2.3/d0fde103346cf89a (running kernel: 7.0.6-2-pve)

Panic logs of the guest here:
Code: 
[153361.974474] Kernel panic - not syncing: Host injected async #PF in kernel mode
[153361.978993] CPU: 4 UID: 0 PID: 394648 Comm: cut Tainted: P           O        7.0.6-2-pve #1 PREEMPT(lazy)
[153361.982973] Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
[153361.985479] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
[153361.989287] Call Trace:
[153361.990120]  <TASK>
[153361.990880]  dump_stack_lvl+0x5f/0x90
[153361.994809]  dump_stack+0x10/0x18
[153361.996472]  vpanic+0x227/0x460
[153361.998465]  panic+0x67/0x70
[153361.999692]  ? early_xen_iret_patch+0xc/0xc
[153362.001405]  __kvm_handle_async_pf+0xa6/0xb0
[153362.004612]  exc_page_fault+0xc4/0x1c0
[153362.007194]  asm_exc_page_fault+0x27/0x30
[153362.009383] RIP: 0010:rep_stos_alternative+0x40/0x80
[153362.011983] Code: c9 75 f6 c3 cc cc cc cc 48 89 07 48 83 c7 08 83 e9 08 74 ef 83 f9 08 73 ef eb de 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89 47
[153362.018863] RSP: 0018:ffffd4124d003b80 EFLAGS: 00050202
[153362.021286] RAX: 0000000000000000 RBX: 00005d9ccae750a0 RCX: 0000000000000f60
[153362.023799] RDX: 00005d9ccae752c0 RSI: 0000000000000000 RDI: 00005d9ccae750a0
[153362.026384] RBP: ffffd4124d003bc0 R08: 0000000000000000 R09: 0000000000000000
[153362.029068] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8c4a1c0e8918
[153362.031643] R13: 0000000000000003 R14: 00005d9ccae74000 R15: 0000000000000000
[153362.034185]  ? elf_load+0x243/0x270
[153362.037219]  load_elf_binary+0x6b5/0x1930
[153362.039304]  ? load_misc_binary+0x22f/0x340 [binfmt_misc]
[153362.041719]  bprm_execve+0x2d1/0x590
[153362.043742]  do_execveat_common.isra.0+0x18d/0x1a0
[153362.045208]  ? do_getname+0x52/0x1e0
[153362.046602]  __x64_sys_execve+0x3e/0x70
[153362.048074]  x64_sys_call+0xc63/0x2390
[153362.050738]  do_syscall_64+0x10b/0x14e0
[153362.053223]  ? count_memcg_events+0xeb/0x190
[153362.056182]  ? handle_mm_fault+0x254/0x370
[153362.058782]  ? do_user_addr_fault+0x2f8/0x820
[153362.060927]  ? irqentry_exit+0xb2/0x600
[153362.062609]  ? clear_bhb_loop+0x50/0xa0
[153362.065034]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[153362.067099] RIP: 0033:0x705059bf5e17
[153362.068442] Code: Unable to access opcode bytes at 0x705059bf5ded.
[153362.071103] RSP: 002b:00007fff3a2179c8 EFLAGS: 00000202 ORIG_RAX: 000000000000003b
[153362.075405] RAX: ffffffffffffffda RBX: 000058bd1b6d57a8 RCX: 0000705059bf5e17
[153362.082192] RDX: 000058bd1b6d5728 RSI: 000058bd1b6d5708 RDI: 000058bd1b6d57a8
[153362.084954] RBP: 000058bd1b6d5708 R08: 00007fff3a219e51 R09: 0000000000000001
[153362.087429] R10: 000058bcfc2f2987 R11: 0000000000000202 R12: 000058bcfc2e8102
[153362.089892] R13: 00007fff3a217ab8 R14: 0000000000000002 R15: 000058bd1b6d5728
[153362.092356]  </TASK>
[153362.094022] Kernel Offset: 0x32400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[153362.101907] ---[ end Kernel panic - not syncing: Host injected async #PF in kernel mode ]---
 
You must log in or register to reply here.
Top Bottom
Back