Bond creation, Linux Bridge Binding concepts, help needed

[ieronymous]

Hello

After reading the wiki, seen some videos and trying to accomplish that myself found me unaware of t he concept behind it still need to understand it because I need to use it. So what I d like to do is .... I have one internet connection (I am mentioning it because of a question below) and proxmox is configured upon the one port of the machine with 192.168.100.30 Gateway is my router's ip 192.168.100.1 in a /24 network. On that machine I have a NIC card with 4 ports and want to use it as one connection upon which an SQL server will be connected. Before that though I ve created the bond without IPv4 and Gateway settings, only filled in the 4 slaves like lets say eth0 eth1 eth2 eth3 and mode balance-rr (later on with a managed switch which wll be lacp compatible I will change the mode here as well to LACP)

At this point comes my first question. What is the reason to fill in in the bond creation an ipv4 and gateway address (well I think you cant specify gateway since there is already the default one been assigned during installation of proxmox) since you cant assign a VM a network bond but a linux bridge?
In addition to that if I change the MTU to 9000 is like enabling jumbo frames? (probably I need to enable it to the switch also)

After that I created a new linux bridge vmbr1 and for Bridge port i only entered the bond0. In this step I have the following questions
-It doesnt let me assign a different gateway with the message gateway: Default gateway already exists on interface 'vmbr0' which is true of course, but what is the point to
have a cell to fill in an ip address for. Also assigning an IP4 address in a different segment than vmbr0 (for example 192.168.200.30) it doesnt make a difference since the VM which has as a network device the newly created vmbr1, still has the same ip address like before when using vmbr0 192.168.100.something and not 192.168.200.something.

I know that some of you might think that I dont have essential network knowledge therefore my questions on that subject, but since I struggle with many things at the same time because there is a time pressure also i d like some help clarify it than just copy-paste a link with tons of stuff to read again.

PS1 The Vm inside (which still has after vmbr change to the new one, the same ip 192.168.100.something) still is able to ping the 192.168.200.something even if in a different network segment.

PS2 Now that I come to think about it bonding and bridging doesnt provide isolation to the VMs, VLans do that.

PS3 Also tried since in other older versions of proxmox been done, to set an ip to the bond 192.168.200.10/24 and then create a linux bridge and assign virtual port bond0. I cant since I get the message <<iface bond0 - ip address can't be set on interface if bridged in vmbr1 (500)>>

Any help would be much appreciated

 

[ph0x]

At this point comes my first question. What is the reason to fill in in the bond creation an ipv4 and gateway address (well I think you cant specify gateway since there is already the default one been assigned during installation of proxmox) since you cant assign a VM a network bond but a linux bridge?

If you want the Proxmox host to be reachable over the NIC or bond, you assign an address. If you have other plans, like defining a bridge on the bond, you let the field empty.

In addition to that if I change the MTU to 9000 is like enabling jumbo frames? (probably I need to enable it to the switch also)

Yes and yes.

-It doesnt let me assign a different gateway with the message gateway: Default gateway already exists on interface 'vmbr0' which is true of course, but what is the point to
have a cell to fill in an ip address for. Also assigning an IP4 address in a different segment than vmbr0 (for example 192.168.200.30) it doesnt make a difference since the VM which has as a network device the newly created vmbr1, still has the same ip address like before when using vmbr0 192.168.100.something and not 192.168.200.something.

As stated above, if you want the host to have a address in the bridge, fill the field, if not then not.
Different NICs have to have a different subnet (or VLAN, which usually results in the same), or you won't have much fun in your network.
A route into that network will automatically be created but the gateway only can be defined once (on vmbr0 usually, but that can be changed as well) for the default route.

PS1 The Vm inside (which still has after vmbr change to the new one, the same ip 192.168.100.something) still is able to ping the 192.168.200.something even if in a different network segment.

See above.

PS2 Now that I come to think about it bonding and bridging doesnt provide isolation to the VMs, VLans do that.

See above.

Hope that helps. You should define another VLAN/subnet for the bond/bridge and then decide if it should only be used for VMs (leave address blank then) or also for the host (enter an address then).

 

[ieronymous]

@ph0x​

thank you for your quick answer but nope I didnt get it, Can you make a gui like example with numbers like I did?

If you want the Proxmox host to be reachable over the NIC or bond, you assign an address.

It is reachable from the pc's port why would I want to be accessible from the quadrant port also? I didn t ask something relevant in my post (at least that is what I think) But lets say I wanted to be reachable how? Bonds cant be assigned to VMs. Also if I set an Ip to the bond then that bond cant be assigned a to a bridge with error message <<iface bond0 - ip address can't be set on interface if bridged in vmbr1 (500)>> So in general what is the point of creating jsut a bond without possibility to assign it somewhere afterwards

 

[ph0x]

It is reachable from the pc's port why would I want to be accessible from the quadrant port also?

Maybe you want a big connection to a backup server? Linux doesn't judge, it just gives you opportunities.

I didn t ask something relevant in my post (at least that is what I think) But lets say I wanted to be reachable how? Bonds cant be assigned to VMs. Also if I set an Ip to the bond then that bond cant be assigned a to a bridge with error message <<iface bond0 - ip address can't be set on interface if bridged in vmbr1 (500)>> So in general what is the point of creating jsut a bond without possibility to assign it somewhere afterwards

The bond without a bridge is usually only usable by the host itself, whatever you want to use it for. If you want VMs to use it, you don't configure addresses or anything on the bond, but go on with the definition of the bridge on it.

 

[ieronymous]

Maybe you want a big connection to a backup server?

You mean like a cheap alternative of a 10gb connection? Then of course, the Backup Server should also have another 4 ports to go 1 -1 with proxmox or 2 trunks on the same switch, 4 for proxmox and 4 for the backup server (if they use the switch as intermediate connection). Ok that would be a case of using it (even though I dont quite get how exactly you use it, which tab / options to fill and where etc)

This conversation it is like Greek Socrates... youre trying toborn out of me the answer. At least you gave me a straight one <<If you want VMs to use it, you don't configure addresses or anything on the bond, but go on with the definition of the bridge on it>> This is a knowledge I can use.

So we skip the bond part and leave it empty with only the ports (so we are at the case I am mentioning at the top of my post since this was how I initially configured it) and we go to the bridge part. The outcome to the VM with that bridge would be the same if i leave ipv4 empty or fill it with any address I can imagine. It is like the VM doesnt care about the settings of the bridge.

It seems to me that the way I configured it at first place and checked it also is to have the VM to use another physical card with 4 ports and even if the vmbr0 goes down the VM inside the Proxmox still has net access (at least I tested that). So for this reason i can get away with no ip on the bridge side also right?

 

[ph0x]

Think of the bridge like of a small network switch. Proxmox itself is the hardware that runs it but can also be accessible through it, depending on if you defined an address in the ipv4 field of the bridge or not.

And yes, as long as vmbr1 is connected, any other connection/bridge can fail without the VMs on vmbr1 losing connection.

 

[ph0x]

But coming back to the two subnets. Now that I think about it, the two vmbr might still be able to work on the same subnet as long as Proxmox itself has only one address in it. I didn't test it but it could work.

 

[ieronymous]

But coming back to the two subnets. Now that I think about it, the two vmbr might still be able to work on the same subnet as long as Proxmox itself has only one address in it. I didn't test it but it could work.

Thanks again for your effort explaining. Most of the parts has been shed with some light now except the part where you give the vmbr an ip address. I dont get why to bother giving one when the VM inside still takes a different ip range from that you specified on the bridge.

My thoughts about that are, that the bridge doesnt have a dhcp server upon it so it cant give the VM inside an Ip according to the range it has. Since the gateway resides at the router side (in case of hardware equipment) or software (pfsense) it is responsible for handing out an ip and it will always be of the 192.168.100. range no matter what ip address the bridge has. So in order for the VM to have different address segment inside a VLAN should be created (of course I dont know the correct order like bond-VLAN-bridge or bond-bridge vlan or vlan-bond-bridge).
If I am right about the above I think I got it now (at least what I did for my use case scenario)

 

[ph0x]

I dont get why to bother giving one when the VM inside still takes a different ip range from that you specified on the bridge.

You're right, the ip ranges have to fit, also when giving the host an address.

My thoughts about that are, that the bridge doesnt have a dhcp server upon it so it cant give the VM inside an Ip according to the range it has. Since the gateway resides at the router side (in case of hardware equipment) or software (pfsense) it is responsible for handing out an ip and it will always be of the 192.168.100. range no matter what ip address the bridge has.

That's correct, I guess.

So in order for the VM to have different address segment inside a VLAN should be created (of course I dont know the correct order like bond-VLAN-bridge or bond-bridge vlan or vlan-bond-bridge).

The usual order is bond-bridge-vlan. Remember though, the VLAN only has to be created if the host itself needs an address. If it's only for the VMs, you can check the box VLAN-aware and add the tag inside the network card definition in the VM's hardware tab without creating a VLAN.

 

[ieronymous]

You're right, the ip ranges have to fit, also when giving the host an address.


That's correct, I guess.


The usual order is bond-bridge-vlan. Remember though, the VLAN only has to be created if the host itself needs an address. If it's only for the VMs, you can check the box VLAN-aware and add the tag inside the network card definition in the VM's hardware tab without creating a VLAN.

Thank you for your help