Multisegment separation

[Ojars Krumins]

Customer desires a configuration of VM's in the cluster where few limitations should be taken in account.
Server that hosts VM's have 4 interfaces (eno1 eno2 eno3 eno4) bonded together in (bond0). Host server should be accessible from dedicated administration network C class segment (f.e. 192.168.50.x/25/.1) with vlan tag 50. VM's hosted on this server should be able to have addresses from literally any network segment in the company. Let's say, 2 VM's from segment 10.20.0.x/22/.1 where vlan tag is 3, 3 VM's from segment 172.16.60.0/24/.1 where vlan tag is 10 etc.
Desired status is that all VMs go through that bond0 while host itself does not belong to any network passed through. Mean host does not have ip address in 10.20.0 network, nor 172... At IP level from host console you should not be able to ping/telnet/ssh addresses of VM's while they are accessible from outside.

Is it possible to build something similar? If yes, then how? I did not found config like I need to make but similar did not work for me.

 

[LnxBil]

Is it possible to build something similar? If yes, then how? I did not found config like I need to make but similar did not work for me.

Should work. Just give your bridge with bond0 no IP, just a vland IP from your management lan. If you have 4 NICs and all are bonded, you also need to have your management interface over that bond, because it is your only network. VLAN configuration of VMs is done directly in the network configuration of the VM.

 

[Ojars Krumins]

Problem I hit is that system does not permit to link more than one bridge to bond when I need few of them. I tried to create virtual interfaces over single bond, but then networking refused to start.

One of configs I tried (but it did not work) was such:

auto lo
iface lo inet loopback

iface enp0s20u13u5 inet manual

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto bond0
iface bond0 inet manual
slaves eno1 eno2 eno3 eno4
bond_miimon 100
bond_mode 802.3ad
bond-downdelay 200
bond-updelay 200

#Management
auto bond0.1
iface bond0.1 inet manual
vlan-raw-device bond0

#VM_subnet
auto bond0.2
iface bond0.2 inet manual
vlan-raw-device bond0


auto vmbr0
iface vmbr0 inet static
address 192.168.50.100
netmask 255.255.255.128
gateway 192.168.50.1
network 192.168.50.0
broadcast 192.168.50.128
bridge_ports bond0.1
bridge_stp off
bridge_fd 1
bridge_hello 2
bridge_maxage 12

auto vmbr1
iface vmbr1 inet static
address 0.0.0.0
bridge_ports bond0.2
bridge_stp off
bridge_fd 9
bridge_hello 2
bridge_maxage 12

 

[LnxBil]

Why would you need more than one bridge? Your management interface does not need a bridge to work.

 

[Ojars Krumins]

I did not find how to push management interface through bond0 and have it for bridge too. The only config that works for me is this:

auto bond0
iface bond0 inet manual
slaves eno1 eno2 eno3 eno4
bond_miimon 100
bond_mode 802.3ad
bond-downdelay 200
bond-updelay 200

auto vmbr0
iface vmbr0 inet static
bridge_ports bond0
bridge_stp off
bridge_fd 1
bridge_hello 2
bridge_maxage 12
address 192.168.50.100
netmask 255.255.255.128
gateway 192.168.50.1

In such case VM with address from 192.168.50.0/25 range have access to "outside world" but any other are cut off.
"Somewhere deep" there is VLAN tagging and switch say's using vlan id 50 for port group however this tag value is not specified in config. Port group have many other vlan id's associated, but first need to fix with at least one.

 

[LnxBil]

You can also create vmbr0.50 for the management network. Your switch has to pass the information on tagged and untagged vlans through. Maybe talk to the network staff.

 

[Ojars Krumins]

Proxmox gui does not allow to create a vmbr0.50 for management network. If I create it manually, when pinging gateway I get Destination host unreachable.

vmbr config now look so. Might be wrong.

auto vmbr0
iface vmbr0 inet manual
bridge_ports bond0
bridge_stp off
bridge_fd 1
bridge_hello 2
bridge_maxage 12


auto vmbr0.50
iface vmbr0.50 inet static
address 192.168.50.100
netmask 255.255.255.128
gateway 192.168.50.1

So it look in shell

# ping 192.168.50.1
PING 192.168.50.1 (192.168.50.1) 56(84) bytes of data
From 192.168.50.100 icmp_seq=1 Destination Host Unreachable

 

[LnxBil]

Without the vlan stuff, you can ping 192.168.50.1? Maybe you have untagged VLAN 50 on your network, so that you do not need to tag it again.