[SOLVED] Network traffic not getting through nested proxmox host

[mlgim]

Hi
Setting Proxmox VE up as a test for 2 customers, who are considering this option on RMOs.
I've created 2 VMs in my vSphere environment, and installed these as Proxmox hosts in a cluster - works like a charm.
(The Proxmox hosts get their IP info from DHCP server in our demolab.)

But when I create Proxmox VMs and LXCs, I cannot use DHCP - they're unable to get a DHCP address.
I tried setting a static IP configuration. I can ping localhost and the Proxmox host running the VM, but nothing beyond that.

This is probably because it is a nested install, but I'd really like to get it working, as I can then spin-up the demo whenever I need it.

The Proxmox hosts (vSphere VMs) are using VMXNET3 vNICs and IP information assigned via DHCP
Here is the /etc/network/interfaces content:

auto lo
iface lo inet loopback

iface ens160 inet manual

auto vmbr0
iface vmbr0 inet static
address 1xx.8.2.40/21
gateway 1xx.8.0.1
bridge-ports ens160
bridge-stp off
bridge-fd 0

It is probably something simple - I'm not used to Linux bridges
Thanks for any help

 

[Stoiko Ivanov]

PVE in general does not work with dynamically assigned addresses (DHCP) - to be more precise it needs to be able to resolve it's own hostname!
(i.e. `ping -c 1 $(uname -r)` needs to output an ip-address configured on the PVE host)

however:

iface vmbr0 inet static
address 1xx.8.2.40/21
gateway 1xx.8.0.1
bridge-ports ens160
bridge-stp off
bridge-fd 0

this is a statically configured interface (if you installed via the PVE iso it does dhcp but uses the IP and configures it statically)

now for the problem of guests not being able to reach the network - this should work - however usually with VMware you would need to enable promiscous mode on the NIC of the PVE VM (in the bridged configuration each guest is connected on layer 2 to the outside - i.e. its mac-address is visible to VMware and this is usually not allowed)

see https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_network_configuration for further steps and explanations

I hope this helps!

 

[mlgim]

Thank you so much for swift help, Stoiko. You were absolutely right. I had enabled promiscuous mode but not forged transmits. When I did, everything works like a charm. Here are the settings needed on the vSphere PortGroup:



Thanks again

 

[Stoiko Ivanov]

Glad that worked!
Thanks for sharing the screenshot - this will certainly help others with a similar task!
Please mark the thread as 'SOLVED'

 

[setuid]

It sounds like these instructions used to work for previous Proxmox versions, but no longer works for current (6.3) versions of VE.

I've tried all of the various suggestions of setting the vnic/pnic to promiscuous inside VE, my ESXi host has always allowed promisc/forged/mac changes and other nested virt products work inside that same install, but not Proxmox.

The settings in mlgim's screenshot above, are similar/model my own settings, and it does not work if Proxmox is installed as a guest under ESXi, and then attempts to install its own guests (ESXi host -> Proxmox VE 6.3 VM -> guest VM (no DHCP, no networking).



Also, the docs linked above by Stoiko, suggest installing the 'ifupdown2' package, but attempting to do so on VE 6.3, forces a removal of almost all of the core proxmox-ve packages, breaking the hypervisor. The two conflict with each other, so you have to choose a working Proxmox VE 6.3, or install ifupdown2, and have a networked Debian VM, without Proxmox.



Are there any other solutions that work with current Proxmox versions? Or more-current docs, that work with the current versions of Proxmox VE?

 

[Stoiko Ivanov]

The two conflict with each other, so you have to choose a working Proxmox VE 6.3, or install ifupdown2, and have a networked Debian VM, without Proxmox.

This sounds like your sources.list in PVE is not setup correctly:
https://pve.proxmox.com/wiki/Package_Repositories

If the ESXi does allow multiple mac-addresses from one VM-nic then PVE with a bridged setup should work without problems.

 

[Optic00]

I am having the same problem with Proxmox 6.3, vmnetwork and vswitch on esxi pretty much allowing everything (same settings as screenshot above from setuid), yet a vm on the nested proxmox does not get an ipv4 via DHCP. Any suggestions would be much appreciated!

 

[Optic00]

additional observations: if i put a static IP on a LCX Container i cannot ping outside my network (for example 1.1.1.1), i can ping anything in my LAN though! after pinging anything in my LAN i can suddenly also PING outside my network. Router is Opnsense. I will try to replicate this issue on another site with another Router and ESXi Server.

 

[Optic00]

i have found the solution/cause: when using a vswitch with more than 1 NIC breaks something on a nested Proxmox install (on esxi) and its (pve) lxc/vms. It doesn't matter what is chosen for traffic distribution on the vswitch or if it (usage of multiple NICs) is deactivated on a port group that Proxmox is on. It will break networking.