For a project, I need to run testing in QEMU with an emulation device. Our CI/CD environment is based on awesome PVE, if possible I'd like to integrate this testing into the current environment too.
So I need to execute QEMU/KVM in an LXC on PVE, but I ran into a problem. The QEMU showed:
```
Could not access KVM kernel module: Operation not permitted
qemu-system-x86_64: failed to initialize KVM: Operation not permitted
```
Even though, executing QEMU without `--enable-kvm` in the LXC is OK, except terrible extremely slow.
Is the above a possible idea?
If YES, do you know what is going on in my situation? And there is a doubt, should I mount the KVM module in the Operating system of HOST, LXC, or both?
The following are my settings and detailed information.
Script:
```
# qemu-system-x86_64 \
-m 4G \
-smp 8 \
-enable-kvm \
-usb -device usb-tablet \
-drive file=windows-10.qcow2,format=qcow2
```
Settings in HOST for the lxc container:
1. lxc config.
```
root@amd-r9-x3900:~# cat /etc/pve/lxc/105.conf
arch: amd64
cores: 20
features: mount=nfs,nesting=1
hostname: devpc-ubuntu
memory: 20480
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=22:22:A3:78:B2:CE,ip=dhcp,type=veth
net1: name=eth1,bridge=vmbr1,hwaddr=96:23:7C:54:6A:aa,ip=dhcp,type=veth
ostype: ubuntu
parent: setup_essential_tools
rootfs: local-storpool:subvol-105-disk-0,size=600G
swap: 4096
lxc.cgroup.devices.allow: c 10:232 rwm
```
2. The CPU supports SVM.
```
root@amd-r9-x3900:~# lscpu | grep svm
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibpb stibp vmmcall sev_es fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca
```
3. The nested is ENABLED
```
root@amd-r9-x3900:~# cat /sys/module/kvm_amd/parameters/nested
1
```
4. The KVM module is there.
```
root@amd-r9-x3900:~# lsmod | grep kvm
kvm_amd 114688 42
kvm 823296 1 kvm_amd
irqbypass 16384 19 kvm
ccp 94208 1 kvm_amd
root@amd-r9-x3900:~#
```
So I need to execute QEMU/KVM in an LXC on PVE, but I ran into a problem. The QEMU showed:
```
Could not access KVM kernel module: Operation not permitted
qemu-system-x86_64: failed to initialize KVM: Operation not permitted
```
Even though, executing QEMU without `--enable-kvm` in the LXC is OK, except terrible extremely slow.
Is the above a possible idea?
If YES, do you know what is going on in my situation? And there is a doubt, should I mount the KVM module in the Operating system of HOST, LXC, or both?
The following are my settings and detailed information.
Script:
```
# qemu-system-x86_64 \
-m 4G \
-smp 8 \
-enable-kvm \
-usb -device usb-tablet \
-drive file=windows-10.qcow2,format=qcow2
```
Settings in HOST for the lxc container:
1. lxc config.
```
root@amd-r9-x3900:~# cat /etc/pve/lxc/105.conf
arch: amd64
cores: 20
features: mount=nfs,nesting=1
hostname: devpc-ubuntu
memory: 20480
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=22:22:A3:78:B2:CE,ip=dhcp,type=veth
net1: name=eth1,bridge=vmbr1,hwaddr=96:23:7C:54:6A:aa,ip=dhcp,type=veth
ostype: ubuntu
parent: setup_essential_tools
rootfs: local-storpool:subvol-105-disk-0,size=600G
swap: 4096
lxc.cgroup.devices.allow: c 10:232 rwm
```
2. The CPU supports SVM.
```
root@amd-r9-x3900:~# lscpu | grep svm
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibpb stibp vmmcall sev_es fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca
```
3. The nested is ENABLED
```
root@amd-r9-x3900:~# cat /sys/module/kvm_amd/parameters/nested
1
```
4. The KVM module is there.
```
root@amd-r9-x3900:~# lsmod | grep kvm
kvm_amd 114688 42
kvm 823296 1 kvm_amd
irqbypass 16384 19 kvm
ccp 94208 1 kvm_amd
root@amd-r9-x3900:~#
```
