issues with nested proxmox in proxmox

[damo2929]

I am trying to configure nested virtualisation but am getting errors. when I start a nested VM created by proxmox.






this is the configuration of the PVE system

root@wlsc-pxmh01:/etc/pve/qemu-server# cat 10037.conf
agent: 1,fstrim_cloned_disks=1
args: -object sev-guest,id=sev1,cbitpos=47,reduced-phys-bits=1 -machine memory-encryption=sev1
balloon: 0
bios: ovmf
boot: order=scsi0;ide2
cores: 6
cpu: host
ide2: isos:iso/proxmox-ve_7.4-1.iso,media=cdrom,size=1096544K
localtime: 0
machine: q35
memory: 65536
meta: creation-qemu=7.2.0,ctime=1682323977
name: TST-PHM01
net0: virtio=AE:AE:3E:2C:C7:49,bridge=devacces,queues=8
net1: virtio=AE:AE:42:34:2F:A9,bridge=Trunk
net2: virtio=AE:AE:8A0:24:F5,bridge=devtrans,mtu=9000,queues=8,rate=300
numa: 0
ostype: l26
scsi0: testlab:vm-10037-disk-1,backup=0,cache=unsafe,discard=on,size=50G,ssd=1
scsi1: testlab:vm-10037-disk-2,backup=0,cache=unsafe,discard=on,size=500G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=6cc5659c-18d8-417b-9866-9a7be505c8a9
sockets: 2
tags: dev
vmgenid: bf5e3775-4365-457b-a437-6a191ded7308


inside that system
root@tst-phm01:~# egrep --color -i "svm|vmx" /proc/cpuinfo | uniq -c
12 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt lbrv nrip_save tsc_scale vmcb_clean pausefilter pfthreshold v_vmsave_vmload vgif umip rdpid arch_capabilities


but the kernel show
[ 6.411194] systemd[1]: Detected virtualization kvm.
[ 7.227690] kvm: no hardware support


how do I enable hardware support in the proxmox bios ?

 

[Dunuin]

See: https://pve.proxmox.com/wiki/Nested_Virtualization#Example:_PVE_hosts_a_PVE_guest_hypervisor

Or disable "KVM hardware virtualization" in the VMs options.

 

[damo2929]

found the issue and its related SEV as disabling SEV the nested systems work. As such I will raise an issue upstream with KVM project and my AMD account manager regarding this feature degradation when SEV is enabled.
btw just disabling KVM acceleration is not really a fix

 

[damo2929]

doing some digging looks like Secure Nested Paging (SEV-SNP) is not in the mainline kernel yet.
AMD Docs

https://www.phoronix.com/news/AMD-SEV-SNP-Arrives-Linux-5.19

 

[Dunuin]

There is an opt-in 6.2 kernel: https://forum.proxmox.com/threads/opt-in-linux-6-2-kernel-for-proxmox-ve-7-x-available.124189/

 

[damo2929]

yeah just did that and nice mess that was made with nonfunctional intel 810 series nic drivers as such I have reverted back to ensure my ceph wasn't degraded for too long, not sure why 6.2 is in the enterprise repo with such issues, will stick with the now old 5.19 for now even if it's not got the SEV-SNP patches in.

 

[Neobin]

yeah just did that and nice mess that was made with nonfunctional intel 810 series nic drivers as such I have reverted back to ensure my ceph wasn't degraded for too long, not sure why 6.2 is in the enterprise repo with such issues, will stick with the now old 5.19 for now even if it's not got the SEV-SNP patches in.

Would be good to report those problems, either in the thread: [1] or on Bugzilla: [2].

Only my guessing: Especially since there might be a good chance, that the (at least) 6.2 kernel might become the new default in PVE 8, which might not be that far away anymore...

Additionally: 5.19 is EOL.

[1] https://forum.proxmox.com/threads/opt-in-linux-6-2-kernel-for-proxmox-ve-7-x-available.124189
[2] https://bugzilla.proxmox.com