VMBR Config vs Port Config

[spetrillo]

Hello all,

My PVE network config is as follows:

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet manual
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 1 20 30

auto vmbr0.1
iface vmbr0.1 inet dhcp

iface enp1s0f0 inet manual

auto vmbr1
iface vmbr1 inet manual
bridge-ports enp1s0f0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 1 10 12

iface enp1s0f1 inet manual

auto vmbr2
iface vmbr2 inet manual
bridge-ports enp1s0f1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 20 25 30

iface enp1s0f2 inet manual

#auto vmbr3
#iface vmbr3 inet static
# address 172.16.2.2/24
# gateway 172.16.2.1
# bridge-ports enp1s0f2
# bridge-stp off
# bridge-fd 0

If I uncomment VMBR3 I lose access to the PVE server itself. VMBR3 is configured to connect to my ISP router, so I can gain Internet access. The LAN IP of the router is 172.16.2.1, thus why I am using the gateway in the config. Why would this cause me to lose full access to my PVE server? Should this config be on the LAN port(enp1s0f2) itself and bypass a VMBR all together?


Thanks,
Steve

 

[norbertm]

You will probably get a "default Gateway" by DHCP on Interface vmbr0.1
Your vmbr3 declares another "default gateway".
Default means: 0.0.0.0: "route all unknown subnets traffic here".
You are probably accessing your PVE from a client which is out of the DHCP subnet.
So you need to add a route entry for your client subnet or change your DHCP Server config.

 

[spetrillo]

Hmmm...

VMBR 1-3 will only be used by one VM, my OPNsense firewall. VMBR 0 will be used by normal VMs that sit in vlan 20 or 30. VMBR0.1 is only for mgmt of my PVE server. Is there a way to configure networking to support what I am trying to do?

 

[norbertm]

If vmbr3 is to be used by the VM only and not by the Proxmox Host there should not be an address+GW setting, I think.

 

[guruevi]

Yes, there are many ways to do this, but you can only have 1 default gateway (you technically can have more, but then you need to define which IP ranges each would handle)

Perhaps make a diagram of your network, VLAN on each of the 3 interfaces you’re physically connecting to and the IP ranges and traffic flows you expect for each network. Right now I think you are confusing yourself. Start with the default gateway, then layer each network on top of that in a way that doesn’t break the others.

 

[spetrillo]

Thanks to you both...yes I should not have defined the static IP and gateway. It will be defined within the OPNsense vm, so only need to have the bridge available and OPNsense config will do the rest. I got confused...