Help with Network Configuration

D

deanfourie

Member
Jan 28, 2023
57
1
8
Hi,

I am setting up Proxmox on a new machine with 6x NICS. I am getting a little confused by the networking setup.

NIC1 will be setup as a TRUNK to allow 3 vlans, vlan 10, 20 and 30.

The remainder of the interfaces will just be standalone interfaces and will be assigned using access ports on the switch for the appropriate vlan.

I have created a separate bridge for each interface, and enabled vlan-aware on vmbr0, and allowed vlans 1-4094

the config looks like this for the trunk interface and bridge.

For some reason, this setup is not working as expected. I cannot ping some interfaces and it appears I can ping other interfaces IP addresses over the trunk that are not even in the trunk (as if all interfaces are bridged.)

Also, does the bridge-fd 0 option need to be unique for each bridge?

Any ideas?

Thanks

Code: 
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!


auto lo
iface lo inet loopback


iface enp1s0f0 inet manual


iface enp1s0f1 inet manual


iface enp0s31f6 inet manual
        post-up /sbin/ethtool -k enp0s31f6 tso off gso off
iface enp1s0f2 inet manual


iface enp1s0f3 inet manual


iface enp2s0 inet manual
       post-up /sbin/ethtool -k enp2s0 tso off gso off


auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp1s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 1-4094
#TRUNK


auto vmbr1
iface vmbr1 inet manual
        address 192.168.1.4/24
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
#OPNSense WAN


auto vmbr2
iface vmbr2 inet manual
       bridge-ports enp1s0f1
       bridge-stp off
       bridge-fd 0
LAN5


auto vmbr3
iface vmbr3 inet manual
       bridge-ports enp1s0f2
       bridge-stp off
       bridge-fd 0
LAN4


auto vmbr4
iface vmbr4 inet manual
       bridge-ports enp1s0f3
       bridge-stp off
       bridge-fd 0
#LAN3


auto vmbr5
iface vmbr5 inet manual
        address 172.16.10.5/24
        bridge-ports enp2s0
        bridge-stp off
        bridge-fd 0
#OPNSense LAN


auto vmbr0.100
iface vmbr0.100 inet manual


auto vmbr0.200
iface vmbr0.200 inet manual


auto vmbr0.300
iface vmbr0.300 inet manual


auto vmbr0.400
iface vmbr0.400 inet manual


auto vmbr1
iface vmbr1 inet manual
        bridge-ports vmbr0.200
        bridge-stp off
        bridge-fd 0
#IoT


auto vmbr2
iface vmbr2 inet static
        address 172.16.103.1/27
        bridge-ports vmbr0.300
        bridge-stp off
        bridge-fd 0
#Management


auto vmbr3
iface vmbr3 inet manual
        bridge-ports vmbr0.100
        bridge-stp off
        bridge-fd 0
#LAN


auto vmbr4
iface vmbr4 inet manual
        bridge-ports vmbr0.400
        bridge-stp off
        bridge-fd 0
#TV
 
Last edited:
deanfourie said:
Also, does the bridge-fd 0 option need to be unique for each bridge?
Click to expand...
No

It seems like you made a copy-paste mistake, since in your network configuration there are several lines that use vmbr0 instead of the correct bridge name:
Code: 
iface vmbr0 inet manual
 
Sorry you are right. I actually did not have access to the /etc/network/interfaces as the device was offline, Had to get it when I got home.

I have now updated the ACTUAL config above. I did some more testing tonight and the results are very strange,

I can ping the static IPs that are defined above for the interfaces NOT in the trunk when plugged directly into my laptop.

However, when plugged into the switch, I cannot see them on the network. The switch is a Cisco Catalyst and I have confirmed that the switchports are definitely in the correct access vlan (untagged).

I am starting to think this may be an issue relating to negotiation or something with the cisco switch, as I said I can ping them find when my laptop interface is set static on the same subnet and plugged directly in.

Please could you check my network config as I have never setup interfaces that are not trunked, I have only ever used 1 physical interface with multiple vlans. I want 1 trunk and the rest just assignable interfaces, eg, WAN and LAN for a firewall.

Thanks
 
You have vmbr1/2/3/4 configured twice, so those two configurations probably get merged. Does it work with unique names for the bridges?
 
shanreich said:
You have vmbr1/2/3/4 configured twice, so those two configurations probably get merged. Does it work with unique names for the bridges?
Click to expand...

Oh right,

So remove

Code: 
auto vmbr0.100
iface vmbr0.100 inet manual


auto vmbr0.200
iface vmbr0.200 inet manual


auto vmbr0.300
iface vmbr0.300 inet manual


auto vmbr0.400
iface vmbr0.400 inet manual

However, i would expect this to only effect my VLAN tags right? not the other physical interfaces
 
Yes, you can remove those as well but the actual issue I meant was that you have two blocks for e.g. vmbr1:

Code: 
auto vmbr1
iface vmbr1 inet manual
        address 192.168.1.4/24
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
#OPNSense WAN


[...]

auto vmbr1
iface vmbr1 inet manual
        bridge-ports vmbr0.200
        bridge-stp off
        bridge-fd 0
#IoT

You'd need to rename those to vmbr200 or some other name that doesn't conflict
 
You must log in or register to reply here.
Top Bottom