Nested Virtual Box

[killmasta93]

Hi,
I was wondering if someone else could shed some light, currently i have proxmox with ubuntu 20 virtualized, inside of that ubuntu im going to run cuckoo but i need virtual box in the ubuntu, but when i check the virtual box i see the VT-x greyed out
i have on options KVM hardware virtualization to yes and also check kvm intel to yes

root@prometheus:~# cat /sys/module/kvm_intel/parameters/nested
Y

Thank you

 

[dylanw]

Just to confirm, you enabled nested virtualisation on the host, not the VM, right?
Did you follow our guide [1] to enable nested virtualisation?
Could you post the following?

• Ubuntu VM config: qm config <vmid>
• Proxmox VE version: pveversion -v
• CPU model

[1] https://pve.proxmox.com/wiki/Nested_Virtualization

 

[killmasta93]

Thanks for the reply,whats odd is that i can run KVM within the ubuntu machine i think it might be a virtualbox 6.1 issue as 5.1 seems to work

root@prometheus2:~# pveversion -v
proxmox-ve: 6.3-1 (running kernel: 5.4.73-1-pve)
pve-manager: 6.3-2 (running version: 6.3-2/22f57405)
pve-kernel-5.4: 6.3-1
pve-kernel-helper: 6.3-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-6
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.3-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.5-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-3
pve-cluster: 6.2-1
pve-container: 3.3-1
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-7
pve-xtermjs: 4.7.0-3
pve-zsync: 2.0-4
qemu-server: 6.3-1
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1

32 x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2 Sockets)

root@prometheus2:~# qm config 169
agent: 1
boot: order=scsi0;ide2;net0
cores: 2
ide2: local:iso/ubuntu-20.04.1-desktop-amd64.iso,media=cdrom
memory: 12048
name: sandbox
net0: virtio=6E:F7:39:47:30:AA,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
parent: clean
scsi0: local-zfs:vm-169-disk-0,discard=on,size=128G
scsihw: virtio-scsi-pci
smbios1: uuid=ff5e8d96-f78d-4634-98fe-6749c3157ccc
sockets: 1
unused1: local-zfs:vm-169-disk-1
vmgenid: af3e2cff-cb22-4045-8058-714eef130bec

 

[dylanw]

Thanks for the reply,whats odd is that i can run KVM within the ubuntu machine i think it might be a virtualbox 6.1 issue as 5.1 seems to work

I have just installed virtualbox 6.1 (from APT, via: focal-updates/multiverse) on ubuntu 20.04, with nested virtualisation enabled on my host, and I had no problems setting VT-x. I am unsure as to why it works for you in version 5.1, but not in 6.1. Where did you install virtualbox from (not that it should matter).

 

[Republicus]

I used to be able to pass CPU flags like +VMX to different virtualized CPUs and get nested virtualization to work in PVE. I know specifically that Penryn used to work this way.
Now, I MUST use cpu: host in order to get nested virtualization to work. Since I do not see that in your qm config you may want to try that. It's also noted in the linked guide.

I'm not sure what changed with qemu that those flags are no longer sufficient --- now qemu reports the CPU (Penryn) doesn't support those features when they used to.

 

[killmasta93]

I have just installed virtualbox 6.1 (from APT, via: focal-updates/multiverse) on ubuntu 20.04, with nested virtualisation enabled on my host, and I had no problems setting VT-x. I am unsure as to why it works for you in version 5.1, but not in 6.1. Where did you install virtualbox from (not that it should matter).

Thanks for the reply so im using this guide

https://notes.netbytesec.com/2020/12/cape-sandbox-installation-from-0-to-hero.html

$ sudo apt update
$ sudo apt install virtualbox virtualbox-ext-pack -y
$ sudo virtualbox

 

[killmasta93]

I used to be able to pass CPU flags like +VMX to different virtualized CPUs and get nested virtualization to work in PVE. I know specifically that Penryn used to work this way.
Now, I MUST use cpu: host in order to get nested virtualization to work. Since I do not see that in your qm config you may want to try that. It's also noted in the linked guide.

I'm not sure what changed with qemu that those flags are no longer sufficient --- now qemu reports the CPU (Penryn) doesn't support those features when they used to.

whats odd is that kvm within the ubuntu works but virtual does not really odd

 

[dylanw]

I would still try to set 'cpu: host' as @Republicus suggested, because the default cpu (kvm64) emulates quite an old CPU model, and mostly exists for the sake of guaranteed compatibility between hosts for migragtion.

 

[killmasta93]

Sorry for the late reply, this week im going to try another approach and post back