Help Configure Internal Network for Guest VMs between Host 1 and Host 2

[AJ.]

Hi Guys,

I need help on setting this up.

I have two host running proxmox. Each host has two physical NIC (eth0 and eth1). eth0 is connected to the public network with a vmbr0 brigde. Each VM's are individually assigned public ip. This config is working as it should.

The one im having issues with is the configuration on eth1 and vmbr1. Both proxmox hosts' eth1 are hooked up on the same switch, and I want guest VMs' on either host communicate with each other. what should I do? so far this configuration has failed me. Guest VMs are not communicating with each other on vmbr1 even within the same host.

also note eth1 from host 1 and eth1 from host 2 are pingable from each other.

/etc/netwok/interface is below

FOR HOST 1 -

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 66.x.x.x
netmask 255.255.255.224
gateway 66.x.x.x
dns-nameservers 209.172.41.202 209.172.41.200
dns-search privatedns.com

auto eth1
iface eth1 inet static
address 10.0.0.1
netmask 255.255.255.0

auto vmbr0
iface vmbr0 inet static
address 70.x.x.x
netmask 255.255.255.252
gateway 70.x.x.x
bridge_ports none
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 10.0.0.100
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0


FOR HOST 2 -

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 88.x.x.x
netmask 255.255.255.224
gateway 70.x.x.x
dns-nameservers 209.172.41.202 209.172.41.200
dns-search privatedns.com

auto eth1
iface eth1 inet static
address 10.0.0.2
netmask 255.255.255.0

auto vmbr0
iface vmbr0 inet static
address 90.x.x.x
netmask 255.255.255.252
gateway 90.x.x.x
bridge_ports none
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 10.0.0.200
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0

 

[totalimpact]

I am not an expert on proxmox networking, but I would approach it as it is out of the box, the default config is to not apply an address to the eth* interface, but instead put it on a bridge, and then the bridge brings up the physical interface like so:

auto lo
iface lo inet loopback

auto vmbr0
iface vmbr0 inet static
address 66.x.x.x
netmask 255.255.255.224
gateway 66.x.x.x
dns-nameservers 209.172.41.202 209.172.41.200
dns-search privatedns.com
bridge_ports eth0 #<<<<<<<<<< 
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 10.0.0.100
netmask 255.255.255.0
bridge_ports eth1 #<<<<<<<<<< 
bridge_stp off
bridge_fd 0

auto vmbr2 for a 3rd network

you could use openvswitch in the gui to make it more complex and route between different layers of different subnets... i havent gotten there yet myself, so i couldnt advise on that.

The problem with your config is this: "bridge_ports none" - the "none" means the bridge is not attached to a physical port, not sure how traffic gets out.

 

[AJ.]

Hi totalimpact

Thank you for responding. You are correct, what you mentioned is the way to go. I have actualy gotten it to work already few days ago but nevertheless, thank you for responding! One other thing to note if there are others whos having issues like this. When making changes to '/etc/network/interfaces' and restarting the network service afterwards, make sure to shutdown the VM then start it back again. Vmbr changes does NOT kick in unless you completely shutdown the VM.

 

[AJ.]

Hi totalImpact,

thanks for responding. First off, your solution is correct and works! I actually got it figured out few days ago but nevertheless, thanks for responding. Just a tip to someone that may be having similar issues. When you edit '/etc/network/interfaces' and restarted the network service; please do not forget to shutdown the individual VMs as well, since the network changes does not kick in to the VM unless it is shutdown.