Test environment - Virtualized Proxmox on esxi - network problem

Bojan Pogacar

New Member
Oct 22, 2017
5
1
1
39
Hello!

I have test environment:

vmware esxi -> Proxmox -> VM
All networks are bridged

Proxmox is fully network reachable from all sides.
VM inside Proxmox can ping proxmox node (if using E1000 ot VMXnet3) that resides on but nothing else.
Proxmox can not ping VM.

Firewall is off.
I've tried with virtualbox - no change.

What am I doing wrong or it is not possible at all?

Thank you!
 
Hello!

I have test environment:

vmware esxi -> Proxmox -> VM
All networks are bridged

Proxmox is fully network reachable from all sides.
VM inside Proxmox can ping proxmox node (if using E1000 ot VMXnet3) that resides on but nothing else.
Proxmox can not ping VM.

Firewall is off.
I've tried with virtualbox - no change.

What am I doing wrong or it is not possible at all?

Thank you!

Would like to join the question
 
Hi,

VMWare network is not compatible with linux bridges.
 
Hello, Wolfgang!

Meanwhile I've added a node on virtualbox and migrated some VMs there. Still no network connectivity.

Is there any other solution to make test environment before making a real pruduction?

Thank you, Bojan
 
Proxmox is fully network reachable from all sides.
VM inside Proxmox can ping proxmox node (if using E1000 ot VMXnet3) that resides on but nothing else.
Proxmox can not ping VM.

This is because Proxmox generates a MAC address for each virtual network interface assigned to CTs. If you look at Ethernet frames leaving your Proxmox on ESXi you will notice that frames leaving ESXi are received by destination node but reply frame is dropped by ESXi.
ESXi does this for all incoming frames having a destination MAC address not allocated by ESXi. ESXi doesn't have a MAC address learning mode. OpenVZ doesn't suffer of this drawback. To get Proxmox out of this impasse you can either :

1/ Turn on promiscuous mode on ESXi to the physical Ethernet interface or to the virtual VLAN. Off course this a bad solution because Ethernet traffic to this VLAN/Ethernet will be sent to all ESXi's VM and Proxmox CTs on this ESXi. It's bad for security and performance reasons.
2/ Or make an Ethernet bridge, assign it an IP address, don't put physical Ethernet interface into it but assign it another IP. Then activate both Proxy ARP and IP forward et voilà ! CT's outgoing IP/Ethernet frames will be routed by the host and so will have their source MAC addresses changed to the one of the host network card. This is why reply frames will be accepted by ESXi and routed back to the Proxmox VM

Here is an example with a big tip where I create a CT's subnet inside the main subnet. This way you don't have to propagate the new route to your network. ARP and IP Forward will magically do the job for you !
If overlapping IP subnet hurts you, you can create a CTs IP subnet outside IP subnet of the host interface (ie. not included inside host's IP subnet - intersection of both subnets is null) and propagate its IP route as usual (RIP, OSPF...). In this case Proxy ARP can be switched off

# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto ens192
iface ens192 inet static
address 192.168.4.79
netmask 255.255.255.255
gateway 192.168.4.1
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp
# or if you prefer : post-up echo 1 > /proc/sys/net/ipv4/conf/ens192/proxy_arp

auto vmbr0
iface vmbr0 inet static
address 192.168.4.80
netmask 255.255.255.240
gateway 192.168.4.1
bridge_ports none
bridge_stp off
bridge_fd 0

# cat /etc/sysctl.d/esxi.conf
#
# /etc/sysctl.conf - Configuration file for setting system variables
# See /etc/sysctl.d/ for additional system variables.
# See sysctl.conf (5) for information.
#

# redirection. Some network environments, however, require that these
# settings are disabled so review and enable them as needed.
#
# Do not accept ICMP redirects (prevent MITM attacks)
net.ipv4.conf.all.accept_redirects = 0

# Do not send ICMP redirects (we are not a router)
net.ipv4.conf.all.send_redirects = 0

# Do not accept IP source route packets (we are not a router)
net.ipv4.conf.all.accept_source_route = 0

Of course this also works for IPv6.

You can see that Proxmox host has two IP. One assigned to the outside network interface (192.168.4.79) and one to the Ethernet bridge (192.168.4.80). Both can be used to connect to the Proxmox WebUI. Note here that 192.168.4.80 is also the network address of the IP subnet. Usually this is not allowed. You can choose not to allocate 192.168.4.80 (network address) and 192.168.4.95 (broadcast address).
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!