Nested ESXI 8.0 - Fails with AMD-V implementation is incompatible

[rvargas9]

I'm trying to build a lab to test ESXI 8.0 and learning Proxmox on this process as well.

I was able to install ESXI and started it without issues, but when I create any VM on the ESXI Is where the issues arise.

I already check that my PVE have:

options kvm-amd nested=1 at etc/modprobe.d

Later I added (I had the feeling its not the issue base on this article)

options kvm ignore_msrs=y
options kvm report_ignored_msrs=0

ESXI VM Created have: vmx.allowNESTED = TRUE (This was added as well on the ESXI config)
Also on the VM at Proxmox (ESXI): echo 'args: -machine vmimport=off' >> MACHINEID.conf

What could I be missing? Most articles are related with Intels
Note: I tried installing Proxmox as VM and creating a VM there, all successful without a glitch, so all seems really related with ESXI 8.0 or both a combination of this Proxmox version and ESXI 8.0.

Physical Host Settings:
model : 8
model name : AMD Ryzen 7 2700X Eight-Core Proce
64 GB RAM
1 TB SSD
BIOS have virtualization enable (SVM Mode Enable)

Error:


Below the machine settings for the EXSI





Also the VM created on the ESXI have this valued added:


Other details form the PVE:

root@homevw:~# grep -R "" /sys/module/kvm_amd/parameters
/sys/module/kvm_amd/parameters/sev_es:Y
/sys/module/kvm_amd/parameters/vls:1
/sys/module/kvm_amd/parameters/pause_filter_count_max:65535
/sys/module/kvm_amd/parameters/nrips:1
/sys/module/kvm_amd/parameters/pause_filter_count_grow:2
/sys/module/kvm_amd/parameters/pause_filter_count_shrink:0
/sys/module/kvm_amd/parameters/npt:Y
/sys/module/kvm_amd/parameters/sev:Y
/sys/module/kvm_amd/parameters/vgif:1
/sys/module/kvm_amd/parameters/nested:1
/sys/module/kvm_amd/parameters/dump_invalid_vmcb:N
/sys/module/kvm_amd/parameters/pause_filter_thresh:128
/sys/module/kvm_amd/parameters/pause_filter_count:3000
/sys/module/kvm_amd/parameters/intercept_smi:Y
/sys/module/kvm_amd/parameters/avic:N

 

[tchuyev]

Hi, exact same problem here with an AMD Ryzen 9 5950x.
Did you manage to resolve the issue since your last post?

No problems at all with an Intel CPU (Xeon E5-2680 v3) running a nested ESXi 8.0 hypervisor.
I was even able to passthrough an NVIDIA Tesla M60 GPU card to the VM!

Hope VMware fixes this issue with AMD with ESXi 8.0.
Works fine with ESXi 7.0U3g FYI.

 

[rvargas9]

Hello,

Not really being honest I really need the lab and I didn’t have any spare machine for it so I went back to Windows 11 with VMware Workstation (latest) and from it I created it and works fine.

My goal was to leave this as lab with Proxmox, but seems as you say AMD is no friend for it at all. This is not my first time with issues on AMD.

 

[tchuyev]

Hello,

Not really being honest I really need the lab and I didn’t have any spare machine for it so I went back to Windows 11 with VMware Workstation (latest) and from it I created it and works fine.

My goal was to leave this as lab with Proxmox, but seems as you say AMD is no friend for it at all. This is not my first time with issues on AMD.

Maybe you should have gone with nested vCenter 8.0 and ESXi 7.0 as everthing is working OK with Proxmox?
Baremetal hypervisor is so much more efficient compared to hosted virtualization (type 2 hypervisor).
I'll post again here if I get any progress with ESXi 8.0 in the future

 

[rvargas9]

Last 6.7 I had problems with the passthrough on the ESXI of course because it’s lab and not certified hardware (not a corp ). For that reason I went that way. Next lab machine I build will be an Intel possible and take a little more time looking for compatibility or know issues.

Still let me know any progress

 

[frankmanzhu]

Looks like this is a bug in the latest Vmware workstation 17 as well. I couldn't get it working.
workstation 16 works fine.

 

[tchuyev]

Check this thread:
https://forum.proxmox.com/threads/proxmox-8-breaks-nested-virtualization.130509/#post-595694

 

[tchuyev]

Issue is finally fixed in Proxmox 8.1.x (kernel 6.5.x): VMs can now be created/ran on top of nested ESXi 8.x hypervisors.
Working using PVE No-Subscription repository.
That's great news