nested virt - proxmox (debian9(windows srv 16))

[ibs0d]

Good day.

For academic purposes nested virtualization is required.
host - proxmox 5.4-3 (CPU:12 x Intel(R) Xeon(R) CPU E5-2603 v4 @ 1.70GHz (2 Sockets))
1 layer: debian 9 (CPU: host (nested enable))
2 layer: any linux - all goods
2 layer:

• windows 7-10 (install slow, virtio support installed, boot slow, work slow)
• windows srv 16 (install slow, after, boot not possible)

what is done: change cpu model for 2 layer - no result

what else can try ?

thx.

 

[Stefan_R]

2 Layer nested virtualization is not going to be very fast to begin with. That said, there are assisting technologies available.

For starters, do you have shadow vmcs enabled? (cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs)
Also, did you check that nesting actually works? (grep "vmx" /proc/cpuinfo should show the vmx flag in layer 2/first vm too)

For Windows guests you should also enable Hyper-V enlightenments. Easy way to do this is to run Proxmox as the first guest (layer 2) as well, and then select "Windows" as the second (layer 3) guest OS in there.

 

[ibs0d]

2 Layer nested virtualization is not going to be very fast to begin with. That said, there are assisting technologies available.

For starters, do you have shadow vmcs enabled? (cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs)
Also, did you check that nesting actually works? (grep "vmx" /proc/cpuinfo should show the vmx flag in layer 2/first vm too)

For Windows guests you should also enable Hyper-V enlightenments. Easy way to do this is to run Proxmox as the first guest (layer 2) as well, and then select "Windows" as the second (layer 3) guest OS in there.

vmcs: enabled
Hyper-V enlightenments: enabled

 

[Stefan_R]

I just did some testing on my machine (Xeon E3-1231 v3) with vmcs enabled and I'm not seeing a particularly big performance drop.

Setup is VE on Host, VE as guest, and Windows 10 as guest on either of these. CPU benchmarking (CBr20) shows a drop of about 7% between regular and nested virtualization.

I would check other vectors as well, depending on how the slowness manifests (CPU, memory, storage, display, etc...). Is your storage fast enough? Are you overcommiting one of the virtualization layers?

For more help I would need more details about your setup (storage type, VM configuration in host and guest).

 

[ibs0d]

layer 1: proxmox
layer 2: win srv16, all good

 

[ibs0d]

I just did some testing on my machine (Xeon E3-1231 v3) with vmcs enabled and I'm not seeing a particularly big performance drop.

Setup is VE on Host, VE as guest, and Windows 10 as guest on either of these. CPU benchmarking (CBr20) shows a drop of about 7% between regular and nested virtualization.

I would check other vectors as well, depending on how the slowness manifests (CPU, memory, storage, display, etc...). Is your storage fast enough? Are you overcommiting one of the virtualization layers?

For more help I would need more details about your setup (storage type, VM configuration in host and guest).

storage: ssd

layer 1: proxmox
layer 2: win srv16, all good

update debian 9 from testing repo

layer 1: debian9
layer 2: win srv16, install not possible (os-variant=win2k16)

 

[Stefan_R]

That is then an error with your debian guest, I suggest you look into plain QEMU or libvirt documentation.

 

[ibs0d]

That is then an error with your debian guest, I suggest you look into plain QEMU or libvirt documentation.

yes

proxmox run winsrv16 with "kvm64,+lahf_lm,+s
ep,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer
" but "virsh edit" not support this hv options.

 

[ibs0d]

That is then an error with your debian guest, I suggest you look into plain QEMU or libvirt documentation.

solved
i check libvirt log for winsrv guest:

.....
warning: host doesn't support requested feature: CPUID.01H:ECX.vmx
.....​

change cpu model for win2k16 guest to

mode='host-passthrough'

this solve my problem

thx