PVE host cannot access internet through second bridge

[Qlii256]

I have a PVE host running with two NICs. I only use one at the moment and it has three virtual bridges. NIC1 has an untagged network (home network) and two tagged VLANs (VM network and VM isolated network). The purpose is that I can create VMs that can access my LAN (home network) and create other VMs that can access the internet using the VM network, where they can also access each other through that network. The third network is for VMs that can only access the internet, but not each other.

I have no rules in my gateway/firewall and still run into issues with my configuration. The PVE host is set up correctly AFAIK, as it has two NICs (only one use atm), two VLANs and three (virtual) bridges. When I open a shell on the host and issue the follow command:

ping -I vmbr0 google.com

I get a response from the google servers.

However, when I issue the following command:

ping -I vmbr1 google.com
ping -I vmbr2 google.com

I get no response and the ping failed.

From my LAN (home network) I can access the server on all of its IP addresses (vmbr0, vmbr1 and vmbr2) without problems. From the host, I can NOT access other LAN devices other than through vmbr0. My firewall (Unifi USG 4 Pro) does not have any rules applied to any of the networks, therefor each VLAN should be able to talk to one another. However, there seems to be an issue with the host OS when routing through different bridges other than the default gateway. I've encountered this issue on a VM as well when I pass two NICs to that VM, and I have to edit Netplan to account for multiple gateways each with a lower/higher metric/priority. However, I'm not sure how to do this on the PVE host.

Another issue I'm facing is when I forward (open) port 80 on my firewall to vmbr0, I can access the web interface of the host from my LAN when using my public IP address, however when I forward it to any of vmbr1 or vmbr2, it does not work. This might be related due to the host not having access to LAN through those bridges.

Clearly something is wrong and I'd like to resolve this so that my host can also access internet and LAN through vmbr1 and vmbr2 just like vmbr0. After, I'll create firewall rules on my gateway to force VLANs not being able to talk to each other, other than certain ports/protocols which I'll allow (such as port 80 for the web interface). I suspect this has something to do with routing. It's coming in on vmbr1, but trying to go out on vmbr0?

# configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback iface eno8303 inet manual iface eno8403 inet manual auto eno8303.10 iface eno8303.10 inet manual auto eno8303.20 iface eno8303.20 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.1.130/24 gateway 192.168.1.1 bridge-ports eno8303 bridge-stp off bridge-fd 0 #Home Network auto vmbr1 iface vmbr1 inet static address 192.168.10.100/24 bridge-ports eno8303.10 bridge-stp off bridge-fd 0 #VM Network auto vmbr2 iface vmbr2 inet static address 192.168.20.100/24 bridge-ports eno8303.20 bridge-stp off bridge-fd 0 #VM Network (gateway)

 

[UdoB]

When you ping via vmbr0 the IP (Internet Protocol) knows that there is an exit (gateway) at 192.168.1.1.

How can a ping leaving your host via vmbr1/vmbr2 know where to step forward? You need a router for this task...

 

[Qlii256]

When you ping via vmbr0 the IP (Internet Protocol) knows that there is an exit (gateway) at 192.168.1.1.

How can a ping leaving your host via vmbr1/vmbr2 know where to step forward? You need a router for this task...

Ok, so this is normal behaviour? But what I cannot figure out is why I can go from LAN to the IP of vmbr1 and vmbr2 no problem (because it's going via router?) from my Windows pc. But when I do a port forward on my gateway for say port 8006 to vmbr1 IP, I cannot access it with my public ip (both outside and inside my LAN).

EDIT: It makes more sense to me now, thank you. With the help of the following link I was able to add another routing table and have a default route set-up for vmbr1 (VLAN 10). This seems to be working now! I can now access the web ui using my external IP address, as well as ping outside the vmbr1 using:

ping -I 192.168.10.100 google.com

Note that using the interface name, vmbr1, does not seem to work. Not sure what's up with that?

Also, I'd like to know if this is the correct way to do something like this. I've removed all the bridges and VLANs and added only a vmbr0 that is vlan-aware. I now have only vmbr1 NIC added to each VM and I put the VLAN-tag in there. It's a bit clearer and that way I only need to manage one bridge device.

https://www.thomas-krenn.com/en/wiki/Two_Default_Gateways_on_One_System

 

[Qlii256]

Anyone got any idea why I cannot simply do:

ping -I vmbr1 google.com

while

ping -I vmbr0 google.com

works, and (192.168.10.100 is the IPv4 address used for the vmbr1 bridge)

ping -I 192.168.10.100 google.com

also works?