4 network segments on one machine and VLANs

[jherazob]

Hi! Been fighting with this since last week and it has utterly evaded me so far.

Hosting has been giving us Proxmox machines before, both times they have given us the IP segment for the VMs on the same subnet as the Proxmox machine itself, configuration is a cinch giving the VMs the --ipaddress directly.

This time they gave us a new one, host machine has one IP, and they gave us 3 other small network segments (2 of 32 IPs and one of 64), each on their own VLAN, with their own netmask/gateway, on a second NIC.

Have been following guides all over the net, from the ones in the Wiki (here, here and here) to others everywhere on the web. And still the VMs can't ping anything.

It seems like there isn't an up to date documentation that tells you how to do it in this case, even the Wiki seems to reference mostly very old information.

Can somebody give me some info on how to do this correctly, down to the correct VM network configuration? All i have clear is that these more complex configs use veth and not venet and that bonding may be required (although it didn't seem to help). Once i have one VM online i can take it from there.

The host is running Proxmox 3.3 and the VMs are all going to be Debian 7. If there's more information needed i can give it.

 

[Mr.Holmes]

Hello jherazob,

difficult to say without having more details.

Could be considered better when available:

- your LAN requirements in detail (which VMs resp. containers should use and reach which addresses)

- the (current) /etc/network/interfaces from the host

- the (current) configuration file(s) of the VMs resp. containers

All i have clear is that these more complex configs use veth and not venet and that bonding may
be required (although it didn't seem to help).

Yes, that´s correct. "venet0" is only useful when you have just one IP for everything in the Container.
Btw.: So I assume wyouhave containers and not kvm machines, right?

Another experience: VLAN over LINUX bridges doesn´t work reliable, use rather openvswitch.

Kind regards

Mr.Holmes

 

[jherazob]

Okay, more details
Proxmox Host itself has it's own functional IP, netmask, gateway, the standard configuration. It's right now on vmbr0, with "bridge_ports" set to "eth0"

Machine has assigned 3 network segments, 2 smallish /27 ones (32 IPs) and a bigger /26 one (64 IPs), for a total of 128 IPs. Each of the 3 segments are in their own VLAN.

Now the latest configuration after several equally nonworking variations. We had configured every VLAN with their own vbr interfaces, each with their own "eth1". Here's VLAN 123:

iface eth1.123 inet manual

auto vmbr123
iface vmbr123 inet manual
    bridge_ports eth1
    bridge_stp off
    bridge_fd 0

Then i added a veth to the container ("vzctl set 100 --netif_add eth0,,,,vmbr123 --save" with 100 being the container ID). Afterwards i configured the IP of the container the standard Debian way (IP, netmask, gateway of the given network on /etc/network/interfaces).

Then i reboot everything just in case, enter the VM and try to ping the outside, and it fails.

What am i doing wrong here?

 

[Mr.Holmes]

Hello jherazob

Okay, more details
Proxmox Host itself has it's own functional IP, netmask, gateway, the standard configuration. It's right now on vmbr0, with "bridge_ports" set to "eth0"

Machine has assigned 3 network segments, 2 smallish /27 ones (32 IPs) and a bigger /26 one (64 IPs), for a total of 128 IPs. Each of the 3 segments are in their own VLAN.

My understanding I collected in a short diagram you find as attachment - please verify if it´s correct.



Assuming "yes":

Then i reboot everything just in case, enter the VM and try to ping the outside, and it fails.

What am i doing wrong here?

So, in order to not mix it up, I defined an "outside 0" for the Proxmox´s "Own LAN" and "outside1,2,3" for the 3 extra VLANs. From the above I understood no ping to any "outside" wors at all.


- to "outside 0" it can only work if Proxmox has routing to i enabled and NAT to it (or the "outside 0" participants knows tne VLAN.1,2,3 segments, which is probably not the case)


- to "outside 1,2,3" it should work, but everything has to configured correctly in Proxmox - the part of /etc/network/interfaces is to less information to consider it (you should send the whole file and the config file from container too). Moreover, as already mentioned, it does not work reliably with simple LINUX bridges, use OVS!

kind regards

Mr.Holmes

 

[jherazob]

Sorry for taking so long to answer, things have been rather... complicated around here because of not having this machine working already.

Hello jherazob
My understanding I collected in a short diagram you find as attachment - please verify if it´s correct.
View attachment 2501
Assuming "yes":
So, in order to not mix it up, I defined an "outside 0" for the Proxmox´s "Own LAN" and "outside1,2,3" for the 3 extra VLANs. From the above I understood no ping to any "outside" wors at all.
- to "outside 0" it can only work if Proxmox has routing to i enabled and NAT to it (or the "outside 0" participants knows tne VLAN.1,2,3 segments, which is probably not the case)
- to "outside 1,2,3" it should work, but everything has to configured correctly in Proxmox - the part of /etc/network/interfaces is to less information to consider it (you should send the whole file and the config file from container too). Moreover, as already mentioned, it does not work reliably with simple LINUX bridges, use OVS!
kind regards
Mr.Holmes

It's more like this:


Every IP is a valid internet one, there's no LAN, DMZs, masquerading or stuff like that in this setup, it's all open-facing. The container 100 is just the first of múltiple virtual machines that will be using those IPs. The one in vmbr0 is the IP of the Proxmox host itself, the rest will go to containers. Every network segment is assigned to a VLAN, which basically will distribute it among the containers in it's "row". So every container will have valid IP addresses and will basically be a standalone server as far as the Internet is concerned.

The configuration of the eth0 is working flawlessly, as i'm remotely connected to the server via SSH.

In this moment the configuration of vmbr1 is empty because somebody here in desperation reformatted the Proxmox, but basically having no more clues i'm inclined to try the most basic one in here in the mostly futile hope that it miraculously works.

 

[Mr.Holmes]

Hello jherazob

Every IP is a valid internet one, there's no LAN, DMZs, masquerading or stuff like that in this setup, it's all open-facing.

Sounds quit clear - why are there necessary more (V)LANs? Assign to each container the requested address and that´s it!

The only one thing you should take care of is to make all these addresses routable from/to proxmox host.

Kind regards

Mr.Holmes