Nested ESXi 8.0U2 on PVE 8.2-1

[DrakkarD]

Hello all,

I'm facing a problem regarding nested ESXi 8.0U2on a PVE 8.2-1. The installation of the ESXi went fine, I could set the Management IP from the console, I'm able to ping this management IP from a remote computer, but from the remote computer, I'm unable to connect to the ESXi management UI https://managementip/ui. If I enable ssh on ESXi the same problem, I can't connect on ESXi SSH.

Do you have any idea or advice ?

Below the PVE network settings :

--- /etc/network/interfaces 2025-02-28 18:21:29.479540450 +0100
+++ /etc/network/interfaces.new 2025-03-04 21:31:27.898123849 +0100
@@ -28,30 +28,11 @@
iface wlo1 inet manual

auto vmbr1
-iface vmbr1 inet manual
- bridge-ports none
- bridge-stp off
- bridge-fd 0
-#VLAN_MGT_0000
-
-auto vmbr2
-iface vmbr2 inet manual
- bridge-ports none
- bridge-stp off
- bridge-fd 0
-#VLAN_LCL_0010
-
-auto vmbr3
-iface vmbr3 inet static
+iface vmbr1 inet static
address 192.168.1.253/24
bridge-ports enp45s0
bridge-stp off
bridge-fd 0
#WAN_ACCESS

-auto VLAN_WAN_0050
-iface VLAN_WAN_0050 inet manual
- vlan-id 50
- vlan-raw-device vmbr0
-
source /etc/network/interfaces.d/*


The remote computer have the address 192.168.254.10/24
Attached the ESXi VM settings, and console.

Thanks

 

[VictorSTS]

What does happen exactly when you try to access the webUI of ESXi ?

 

[DrakkarD]

I got an error : ERR_SSL_PROTOCOL_ERROR.
I just don't understand why I got this message, I have already connected to physical ESXi 8.0U2 with the same remote computer without problem.
I even tested with old version of Firefox, where I can bypass certificate check, with this old version I can say that I want bypass the certificate but nothing else happend.

Any idea ?

 

[VictorSTS]

Yes, you need to disable CSO offloading inside ESXi. Don't really know why just yet, found it after 2 days of trial and error and browsing online docs. The idea came from here [1] and the fact that tcpdump shows lots of crc errors typically related to misbehaving offloading in the drivers:

esxcli network nic cso set -e 0 -n vmnic0

To make it persistent on boot:

vi /etc/rc.local.d/local.sh

localcli network nic cso set -e 0 -n vmnic0

I have to use this with both intel e1000e and vmxnet3 as the nic in PVE to make it work. I'm still trying to make the network of the nested VM's running inside ESXi to work correctly. Maybe another option could be to use the alternative drivers for intel nics that is/was available in Fling [2], although I'm not that versed on ESXi nor have access to the needed files

Tested with 7.0 Update 3 and 8.0.0.

I would be very grateful if some ESXi expert could lend us a hand to get this working flawlessly!

[1] https://williamlam.com/2022/07/quic...s-from-community-networking-driver-fling.html
[2] https://williamlam.com/2022/09/vsphere-8-productizes-community-networking-driver-fling-for-esxi.html

 

[DrakkarD]

Man,
You save my days. It works !

Thank you

 

[VictorSTS]

Does the network of the VMs running inside ESXi work correctly for you?

 

[DrakkarD]

Does the network of the VMs running inside ESXi work correctly for you?

Hello,
I'm not actually at this stage, I was trying to install vCenter 8.0U3 directly on PVE. I was able to manage it, but it was more a install on VMware Workstation and then a migration to PVE.
I have just checked but ESXi re again not available on GUI, even if I put the line inside the file /etc/rc.local.d/local.sh. I must check if it's still there. Argh, I just checked, I'm in EFI mode, local.sh doesn't work in this mode.

 

[VictorSTS]

I'm in EFI mode, lacal.sh doesn't work in this mode.

Had no idea about that. I used good old BIOS for the PVE VM I used to install ESXi.

I was trying to install vCenter 8.0U3 directly on PVE.

I strongly advise against installing anything in the PVE host itself. Use another VM to get vCenter running.

 

[DrakkarD]

Had no idea about that. I used good old BIOS for the PVE VM I used to install ESXi.


I strongly advise against installing anything in the PVE host itself. Use another VM to get vCenter running.

I'm not sure to get you there.
I created in PVE a VM for pfSense, 3 VM for 3 ESXi , another one for vCenter. I think it's fine like that ?
I don't want to put a second level of virtualization for the vCenter ( vCenter on ESXi on PVE ).

 

[VictorSTS]

I created in PVE a VM for pfSense, 3 VM for 3 ESXi , another one for vCenter. I think it's fine like that ?

It's fine! I somehow understood that you where to install vCenter in the same host of PVE, not in a VM. Sorry, seems I had a shortcircuit in my brain.