No network access on second bridge

[strachanbrad]

Hello community,

I am having a networking issue I am trying to solve, likely due to me being high enough on the learning curve. I have 2 network bridges configured the default(vmbr0) on eno1 and vmbr1 on eno1. I want vmbr0 dedicated to management and vmbr1 for anything that will be hosted on the internet. However I am unable to access the network from my test VM (fresh Ubuntu 22.04) configured with vmbr1. Oddly though I can reach the VM through ssh from my PC over LAN, and the VM does get a an IP assigned with DHCP. I've attached a network diagram, and some configs. If any further info is need please ask.

Firewall rules are allow all on each interface for testing at the moment
Management net 10.0.2.0/24 (Gateway 10.0.2.1)
WAN_HOST net 10.0.5.0/24 (Gateway 10.0.5.1)


*proxmox* cat /etc/network/interfaces

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0
iface vmbr0 inet static
    address 10.0.2.5/24
    gateway 10.0.2.1
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address 10.0.5.0/24
    bridge-ports eno4
    bridge-stp off
    bridge-fd 0
#WAN_HOST

*proxmox* cat /etc/hosts

127.0.0.1 localhost.localdomain localhost
10.0.2.5 pve1.homelab pve1

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

*proxmox* ip address

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether bc:30:5b:f0:d2:bc brd ff:ff:ff:ff:ff:ff
    altname enp1s0f0
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether bc:30:5b:f0:d2:bd brd ff:ff:ff:ff:ff:ff
    altname enp1s0f1
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether bc:30:5b:f0:d2:be brd ff:ff:ff:ff:ff:ff
    altname enp1s0f2
5: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP group default qlen 1000
    link/ether bc:30:5b:f0:d2:bf brd ff:ff:ff:ff:ff:ff
    altname enp1s0f3
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:30:5b:f0:d2:bc brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.5/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::be30:5bff:fef0:d2bc/64 scope link
       valid_lft forever preferred_lft forever
12: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:30:5b:f0:d2:bf brd ff:ff:ff:ff:ff:ff
    inet 10.0.5.0/24 scope global vmbr1
       valid_lft forever preferred_lft forever
    inet6 fe80::be30:5bff:fef0:d2bf/64 scope link
       valid_lft forever preferred_lft forever

*VM* ip address

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fc:c5:1c:26:27:b6 brd ff:ff:ff:ff:ff:ff
    altname enp0s18
    inet 10.0.5.104/24 metric 100 brd 10.0.5.255 scope global dynamic ens18
       valid_lft 7153sec preferred_lft 7153sec
    inet6 fe80::fec5:1cff:fe26:27b6/64 scope link
       valid_lft forever preferred_lft forever

 

[vesalius]

Happy new year.

you do not have to designate the subnet on vmbr1 with the address 10.0.5.0/24 line. Unless you want to have the Proxmox node web gui available to your proxmox guest using vmbr1, I would remove that line, change iface vmbr1 inet static to iface vmbr1 inet manual, and then do a ifreload -a . If I am mistaken and you do want the Proxmox webgui accessible to the wan_hosts subnet then pick an ip other than 10.0.5.0.

to be clear did you define the Gateway 10.0.5.1 via DHCP or in the ubuntu vm?

 

[strachanbrad]

Happy new year.

you do not have to designate the subnet on vmbr1 with the address 10.0.5.0/24 line. Unless you want to have the Proxmox node web gui available to your proxmox guest using vmbr1, I would remove that line, change iface vmbr1 inet static to iface vmbr1 inet manual, and then do a ifreload -a . If I am mistaken and you do want the Proxmox webgui accessible to the wan_hosts subnet then pick an ip other than 10.0.5.0.

to be clear did you define the Gateway 10.0.5.1 via DHCP or in the ubuntu vm?

Happy New Year!

Thank you for your reply.

I have made the changes you suggested, I have tried that however I was unsure of the difference. And yes your correct I don’t want the webgui accessible via web_host

The gateway is configure via dhcp but I have tried setting it in the VM aswell.

I can’t ping anything still.

I even tried ping -I vmbr1 from proxmox to avail.

 

[vesalius]

Ok, have You double checked the managed switch ports? Not sure of your switch manufacturer nomenclature, but is the switch port connected to eno4 untagged vlan5 and is the switch port connected to pfSense tagged for vlan5?

is the Proxmox firewall unchecked on the vm network interface?

for trouble shooting purposes maybe give the Proxmox node a listening address on vmbr1. Temporarily revert back to iface vmbr1 inet static with address 10.0.5.2 for example, and then do a ifreload -a. After check if you can ping 10.0.5.2 from pfSense. From the switch (if you have network tools available there) can you ping bith 10.0.5.1 and 10.0.5.2?

 

[strachanbrad]

Ok, have You double checked the managed switch ports? Not sure of your switch manufacturer nomenclature, but is the switch port connected to eno4 untagged vlan5 and is the switch port connected to pfSense tagged for vlan5?

is the Proxmox firewall unchecked on the vm network interface?

for trouble shooting purposes maybe give the Proxmox node a listening address on vmbr1. Temporarily revert back to iface vmbr1 inet static with address 10.0.5.2 for example, and then do a ifreload -a. After check if you can ping 10.0.5.2 from pfSense. From the switch (if you have network tools available there) can you ping bith 10.0.5.1 and 10.0.5.2?

Thank you again for the reply.

I have a Dell force10 the port on the switch is untagged vlan 5, eno4 is unaware of any vlans.

I can ping the VM from pfsense, the switch, and my lan and I’m even ssh’d into it. I can even log into the webgui on VLAN5. Only outgoing is the problem.

It must be something small somewhere grrr. I like thoughts though keep ‘em coming if you don’t mind lol.

 

[vesalius]

This sounds like a setup issue with pfSense letting vlan5 traffic to/from wan then Or DNS. Can the vm ping 1.1.1.1 or 8.8.8.8?

 

[strachanbrad]

This sounds like a setup issue with pfSense letting vlan5 traffic to/from wan then Or DNS. Can the vm ping 1.1.1.1 or 8.8.8.8?

Roger that I’ll poke arround more thank you for your time @vesalius