Nested KVM

mo_

Active Member
Oct 27, 2011
401
3
38
Germany
Hi,

I was trying to set up a testing cluster for proxmox inside proxmox, but I ran into this error:

boot.log:
Code:
Tue Mar 18 00:04:54 2014: Loading kernel module kvm-intel.
Tue Mar 18 00:04:54 2014: ERROR: could not insert 'kvm_intel': Unknown symbol in module, or unknown parameter (see dmesg)

dmesg:
Code:
kvm_intel: Unknown parameter `nested'

This is on the host proxmox, not inside the virtual proxmox by the way.
Would it be possible to compile this module with this parameter enabled? Or maybe this has been enabled in 3.2 already? Still running 2.6.32-26 (3.1-24) on this cluster...
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,047
737
133
try kvm-intel instead of kvm_intel.

nesting is not really working great. if you want to try you need to run kernel 3.10 on the host, also a modern intel cpu or better, amd.
 

Fishboneiii

New Member
Jan 15, 2015
7
0
1
Hello I have setup proxmox 3.2 and 3.3 in many environments and I do not have the /sys/module/kvm_intel/parameters/nested file on my system.
Thusly when I set the kvm-intel nested=1 parameter dmesg produces "kvm: unknown parameter nested".
I have tried this in a debian environment with qemu-kvm and the parameter is accepted, but the proxmox hypervisor does not accept it, nor does it have that nested file.
My server definitely supports hardware virtualization and the only issue I run into is attempting to use nested virtualization.
I specifically prefer the proxmox hypervisor over qemu-kvm and would not like to have to switch to qemu-kvm or esxi, there may be something I am forgetting, all research leads to this being the reasonable method to nested virtualization in proxmox.
Does anyone have any idea what this could be, am I required to install another application on top of the hypervisor?

Thank you for your time.
 

Fishboneiii

New Member
Jan 15, 2015
7
0
1
Yes I do, I have an Intel Core i7 860, I accidentally asked with the directory as kvm_amd, that was a typo that I corrected, thank you for your response.
 

mir

Famous Member
Apr 14, 2012
3,553
112
83
Copenhagen, Denmark
Do you see this?
modinfo kvm_intel
filename: /lib/modules/2.6.32-34-pve/kernel/arch/x86/kvm/kvm-intel.ko
license: GPL
author: Qumranet
srcversion: 29DE5AB00C6202D0F0A3ECB
depends: kvm
vermagic: 2.6.32-34-pve SMP mod_unload modversions
parm: bypass_guest_pf:bool
parm: vpid:bool
parm: flexpriority:bool
parm: ept:bool
parm: unrestricted_guest:bool
parm: eptad:bool
parm: emulate_invalid_guest_state:bool
parm: yield_on_hlt:bool
parm: vmm_exclusive:bool
parm: ple_gap:int
parm: ple_window:int

It seems the nested option is not available!!!

Compared with amd:
modinfo kvm_amd
filename: /lib/modules/2.6.32-34-pve/kernel/arch/x86/kvm/kvm-amd.ko
license: GPL
author: Qumranet
srcversion: FCDFEAB2882781860E868BF
depends: kvm
vermagic: 2.6.32-34-pve SMP mod_unload modversions
parm: npt:int
parm: nested:int
 
Last edited:

mir

Famous Member
Apr 14, 2012
3,553
112
83
Copenhagen, Denmark
From one of my hosts:
cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 21
model : 2
model name : AMD Opteron(tm) Processor 3350 HE
stepping : 0
cpu MHz : 2799.860
cache size : 2048 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 16
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core arat cpb npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 5599.72
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro
.......

cat /sys/module/kvm_amd/parameters/nested
1
 

Fishboneiii

New Member
Jan 15, 2015
7
0
1
Do you see this?
modinfo kvm_intel
filename: /lib/modules/2.6.32-34-pve/kernel/arch/x86/kvm/kvm-intel.ko
license: GPL
author: Qumranet
srcversion: 29DE5AB00C6202D0F0A3ECB
depends: kvm
vermagic: 2.6.32-34-pve SMP mod_unload modversions
parm: bypass_guest_pf:bool
parm: vpid:bool
parm: flexpriority:bool
parm: ept:bool
parm: unrestricted_guest:bool
parm: eptad:bool
parm: emulate_invalid_guest_state:bool
parm: yield_on_hlt:bool
parm: vmm_exclusive:bool
parm: ple_gap:int
parm: ple_window:int

It seems the nested option is not available!!!

Compared with amd:
modinfo kvm_amd
filename: /lib/modules/2.6.32-34-pve/kernel/arch/x86/kvm/kvm-amd.ko
license: GPL
author: Qumranet
srcversion: FCDFEAB2882781860E868BF
depends: kvm
vermagic: 2.6.32-34-pve SMP mod_unload modversions
parm: npt:int
parm: nested:int

Yes I do get this exact result except the filename is "2.6.32-32-pve" instead of "2.6.32-34-pve"
When run on the same hardware with qemu-kvm as the hypervisor I notice that the "nested:bool" parameter is present.
This is leading me to believe there is a malfunction in the way I have proxmox configured/installed on my server as opposed to the hardware itself. Pardon me for being meticulous, I am really trying to understand this entirely.
Could it be the proxmox hypervisor kernel's communication with the processor causing this issue?
 

mir

Famous Member
Apr 14, 2012
3,553
112
83
Copenhagen, Denmark
What you see means that the kvm_intel module is not compiled with support for nested virtualization. kernel 3.10 seems like your only option then but remember kernel 3.10 has not support for openvz.
 

Fishboneiii

New Member
Jan 15, 2015
7
0
1
Where would I be able to find that, please.
And thanks a lot for your guidance. I do not require openvz currently.
 

Fishboneiii

New Member
Jan 15, 2015
7
0
1
Thank you very much, you definitely resolved my issue and helped me get a grasp of what was really going on.
 

Fishboneiii

New Member
Jan 15, 2015
7
0
1
Hey all, thanks again. After lots of tinkering I was able to get it all working but my hardware was unable to support it. I have a new server and have created a cluster with the 2. The new server is perfectly capable of nested virtualization utilizing the steps above. I am trying to push an openstack vdi through proxmox which is why I needed nested kvm, if I hadn't mentioned it. Thanks again, I'll be sure to update on anything new I discover to share.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!