cpu=host enables nested virutalization… but is way slower than cpu=kvm64… forum and wiki are contradictory

[safar]

Hi

to get Ubuntu to work under Windows 10 (and avoid the "Please enable the virtual Machine Platform Windows feature and ensure virtualization in the BIOS"), I have to set the cpu type to host.

But then guest if very slow.

Which is expected according to Proxmox staff @Stefan_R : https://forum.proxmox.com/threads/nested-virtualization-slow-with-cpu-type-host.92512/post-403224

But the wiki https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines says "set the CPU type to host, as in theory this will give your guests maximum performance."

Which one is right?

Anyway to get nested virtualization working without losing performance on the guest on which to do virtualization?

thanks

 

[oguz]

hi,

to get Ubuntu to work under Windows 10 (and avoid the "Please enable the virtual Machine Platform Windows feature and ensure virtualization in the BIOS"), I have to set the cpu type to host.

But then guest if very slow.

which guest is slow? ubuntu or windows?
how exactly are you running the ubuntu under windows?

can you post the configuration of your VM from PVE? (qm config VMID) and also your pveversion -v?

lscpu would be also interesting to see. what kind of hardware are you running on?

 

[fragtion]

Same issue here. Ryzen CPU running Windows 10. Proxmox virtualized with VMWare Player with AMD-V/RVI checked on the guest. Nested VM's extremely slow with host setting (especially at boot stage) while putting excessive load on the actual host's CPU

 

[VictorSTS]

Same issue here. Ryzen CPU running Windows 10. Proxmox virtualized with VMWare Player with AMD-V/RVI checked on the guest. Nested VM's extremely slow with host setting (especially at boot stage) while putting excessive load on the actual host's CPU

Just to clarify, are you running Windows 10 host OS + VMWare Player + Proxmox in a VMWare VM + a VM running inside that Proxmox?

 

[fragtion]

Just to clarify, are you running Windows 10 host OS + VMWare Player + Proxmox in a VMWare VM + a VM running inside that Proxmox?

That's correct. May seem overly convoluted in concept, but in theory, assuming nesting worked correctly, the performance impact in practice *should* be minimal I'd imagine, particularly with AMD-V/RVI passthrough coupled with using "host" CPU type for guests? What I'm seeing is another story though - with "host" type it is worryingly slow and buggy. kvm64 at least works OK and with relatively fast boot times... but then I lose all forms of virtualization in the sub-nested guest, if that makes sense! So no Windows Subsystem for Linux/Android for instance..

On a bit of a separate note - I'm considering replacing Windows on my personal gaming/workstation PC, with Proxmox (along with GPU passthrough) as this seems to be an attractive upcoming trend especially for virtualization enthusiasts... However from what I'm seeing here, it seems that's going to be a challenging idea if it turns out that I can't run things like WSL2 or Hyper-V on the Windows "guest" with near-native performance (as those will also be "nested" with the same number of layers)

While the above two scenarios don't seem much different when you think about it, I can easily see someone falling into the same trap taking that approach of replacing Windows with Proxmox on a workstation PC.

So if running say WSL2 or Hyper-V on a virtualized workstation isn't such a bad idea, then why should running Proxmox virtualized on a baremetal Windows Host and then host nested guests with that, be any different/worse of an idea?

Unfortunately I'm the "tail" and my client is the "dog" in this case, so this setup is the best I can do right now. Any ideas or suggestions to optimize things further (such as perhaps replacing VMWare with VirtualBox or Hyper-V in this case if there are any possible benefits) are welcome.

Thanks

 

[safar]

I confirm the issue is still here.

Here is the requested info @oguz :

Spoiler: lscpu

lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              8
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           62
Model name:                      Intel(R) Xeon(R) CPU E5-2687W v2 @ 3.40GHz
Stepping:                        4
CPU MHz:                         3717.246
CPU max MHz:                     4000.0000
CPU min MHz:                     1200.0000
BogoMIPS:                        6800.42
Virtualization:                  VT-x
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        4 MiB
L3 cache:                        50 MiB
NUMA node0 CPU(s):               0-7,16-23
NUMA node1 CPU(s):               8-15,24-31
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:               Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb r
                                 dtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx
                                  est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti
                                 tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts

Spoiler: qm config 133

qm config 133
bios: ovmf
boot: order=sata0
cores: 8
cpu: host
efidisk0: local:101/vm-101-disk-0.raw,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:0b:00.0,pcie=1,x-vga=1
hostpci1: 0000:85:00.0,pcie=1,rombar=0
hostpci2: 0000:0a:00.0,pcie=1,rombar=0
hostpci3: 0000:0d:00.0,pcie=1,rombar=0
ide0: local:iso/virtio-win-0.1.208.iso,media=cdrom,size=543390K
ide2: local:iso/win10-install.iso,media=cdrom,size=5745798K
machine: pc-q35-6.1
memory: 32048
meta: creation-qemu=6.1.0,ctime=1637353072
name: windows
net0: virtio=C2:45:E0:B5:7A:4A,bridge=vmbr0
numa: 0
ostype: win10
sata0: /dev/disk/by-id/ata-Samsung_SSD_870_EVO_500GB_S62BNX0RA13015T,size=488386584K
sata2: local:iso/HBCD_PE_x64.iso,media=cdrom,size=3026566K
scsihw: virtio-scsi-pci
smbios1: uuid=f7b01aa5-8ea0-4325-9d8e-18b02680d3ca
sockets: 2
tablet: 0
usb0: host=045e:07f8,usb3=1
usb1: host=046d:c05b,usb3=1
usb2: host=096e:0201,usb3=1
usb3: host=0557:2221,usb3=1
usb4: host=1edb:da0a,usb3=1
vga: none
vmgenid: 00874467-b14e-46a3-8c3c-a75ca86503b1

Spoiler: pveversion -v

 pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve)
pve-manager: 7.1-7 (running version: 7.1-7/df5740ad)
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-1-pve: 5.13.19-3
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

Spoiler

It is the Windows Guest that is slow when using cpu: host, and the same guest is faster when using cpu: kvm64.

 

[safar]

Issue beign: sub-optimal performance in Windows unless cpu: kvm64
Other guests do not have this issue (Centos, MacOs…)