Nested ESXi - vm's don't start

bobmc

Renowned Member
May 17, 2018
718
122
88
67
I'm trying to run ESXi 7.0.3 under proxmox v8.0.4 on an AMD Ryzen host

ESXi installs OK but I can't get a VM to start. I think the PVE Host is configured ok as I am able to run a VM on a nested PVE instance.

ESXi VM config
balloon: 0 boot: order=sata0;ide2;net0 cores: 6 cpu: host ide2: ProxLib:iso/VMware-VMvisor-Installer-7.0U3n-21930508.x86_64.iso,media=cdrom,size=391216K memory: 8192 meta: creation-qemu=8.0.2,ctime=1696158590 net0: vmxnet3=BE:99:8A:14:E1:34,bridge=vmbr0 numa: 0 ostype: l26 sata0: VMS:vm-1001-disk-0,size=32G sata1: VMS:vm-1001-disk-1,size=100G,ssd=1 smbios1: uuid=4a64aa47-2774-42eb-97c0-5a797c44f33a sockets: 1 vmgenid: a96dd967-7682-4a9c-8e83-ece6da0b286a

There is no error message in the GUI, the VM just fails to start. The tail of the VMKernel warning logs are below

2023-10-01T11:47:48.373Z cpu3:134897)WARNING: UserObj: 5569: vsi_traverse: Unimplemented operation on 0x4540c57e2fa8/SOCKET_VMCI 2023-10-01T11:47:50.900Z cpu3:134927)WARNING: UserObj: 5569: vsi_traverse: Unimplemented operation on 0x4540c57e2fa8/SOCKET_VMCI 2023-10-01T11:47:57.413Z cpu5:134897)WARNING: MemTier: 520: MemTierSummary is not initialized 2023-10-01T11:47:57.413Z cpu5:134897)WARNING: MemTier: 402: MemTierSummary is not initialized 2023-10-01T11:47:57.414Z cpu5:134897)WARNING: cswitch: VLAN_PortGetVLANData:463: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]No vlan data for non dvs ports or ports without port group 0x4000003 2023-10-01T11:47:57.414Z cpu5:134897)WARNING: cswitch: VLAN_PortGetVLANData:463: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]No vlan data for non dvs ports or ports without port group 0x84000004 2023-10-01T11:47:57.414Z cpu5:134897)WARNING: cswitch: VLAN_PortGetVLANData:463: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]No vlan data for non dvs ports or ports without port group 0x4000005 2023-10-01T11:48:00.338Z cpu3:134927)WARNING: MemTier: 520: MemTierSummary is not initialized 2023-10-01T11:48:00.338Z cpu3:134927)WARNING: MemTier: 402: MemTierSummary is not initialized 2023-10-01T11:48:00.338Z cpu3:134927)WARNING: cswitch: VLAN_PortGetVLANData:463: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]No vlan data for non dvs ports or ports without port group 0x4000003 2023-10-01T11:48:00.338Z cpu3:134927)WARNING: cswitch: VLAN_PortGetVLANData:463: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]No vlan data for non dvs ports or ports without port group 0x84000004 2023-10-01T11:48:00.338Z cpu3:134927)WARNING: cswitch: VLAN_PortGetVLANData:463: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]No vlan data for non dvs ports or ports without port group 0x4000005 2023-10-01T11:48:14.688Z cpu3:135251)WARNING: UserObj: 5569: vmkvsitools: Unimplemented operation on 0x4540c57e2fa8/SOCKET_VMCI 2023-10-01T11:48:20.165Z cpu1:131096)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "t10.ATA_____QEMU_DVD2DROM____________________________QM00003_____________" state in doubt; requested fast path state update... 2023-10-01T11:48:33.125Z cpu1:135507)WARNING: UserFile: 2298: rm: Directory changing too often to perform readdir operation (11 retries), returning busy 2023-10-01T11:48:33.129Z cpu1:135507)WARNING: UserFile: 2298: rm: Directory changing too often to perform readdir operation (11 retries), returning busy 2023-10-01T11:48:33.133Z cpu1:135507)WARNING: UserFile: 2298: rm: Directory changing too often to perform readdir operation (11 retries), returning busy 2023-10-01T11:48:33.137Z cpu1:135507)WARNING: UserFile: 2298: rm: Directory changing too often to perform readdir operation (11 retries), returning busy 2023-10-01T11:48:33.143Z cpu1:135507)WARNING: UserFile: 2298: rm: Directory changing too often to perform readdir operation (11 retries), returning busy 2023-10-01T12:20:29.667Z cpu4:131098)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "t10.ATA_____QEMU_DVD2DROM____________________________QM00003_____________" state in doubt; requested fast path state update... 2023-10-01T12:20:29.691Z cpu1:136568)WARNING: World: vm 136568: 8442: vmm0:t1:vcpu-0:Invalid VMCB.

Does anyone know if there's a way to run ESXi nested on Proxmox? Would really be useful for my work currently - thanks
 
vmm0:t1:vcpu-0:Invalid VMCB

Yep, this is a bug and exists in kernel 6, its still not fixed.

Only work around is to boot kernel 5. Its a pain but has been around for a few months.
 
  • Like
Reactions: bobmc
to be honest im suprised no one else has reported it on the forums :)
 
I can confirm running kernel 5.13.19-6-pve resolves the issue and allows nested ESXI to run vm's
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!