Bond + VLAN over multiple interfaces

morik_proxmox

New Member
Jan 2, 2023
16
1
3
Hello experts,
I'm stuck on a problem on which i'd appreciate your help.

I have a server with 6 physical ports. For simplicity lets focus only on three physical ports {(p1, p2), p3, p4}. All ports are access (from a switch perspective) ports.
Code:
p1,p2 - VLAN 140, bonded (balance-xor, non-lacp), default gateway
p3,p4 - VLAN 100, bonded (balance-xor, non-lacp)
p1,p2 are "virtual ports" of the same physical port on a broadcom NIC which uses NPar feature at the firmeware level. Both ports get a different mac address towards hypervisor/guestOS, but present only a single (real) MAC address to peers.

Code:
auto p1
iface p1 inet manual
    mtu 9000

auto p2
iface p2 inet manual
    mtu 9000

auto p3
iface ens4f0np0 inet manual
    mtu 9000

auto p4
iface ens4f1np1 inet manual
    mtu 9000

auto bond0
iface bond0 inet manual
    bond-slaves p1 p2
    bond-miimon 100
    bond-mode balance-xor
    mtu 9000

auto bond4
iface bond4 inet manual
    bond-slaves p3 p4
    bond-miimon 100
    bond-mode balance-xor
    mtu 9000

auto vmbr0
iface vmbr4 inet static
    address 192.168.100.12/24
    bridge-ports bond0
    bridge-stp off
    bridge-fd 0
    mtu 9000

auto vmbr4
iface vmbr4 inet static
    address 192.168.100.41/24
    bridge-ports bond4
    bridge-stp off
    bridge-fd 0
    mtu 9000


Switch configuration is
Code:
p (p1,p2):
description Mgmt VLAN
switchport mode access
switchport access vlan 140

p3,p4:
description Server VLAN
switchport mode access
switchport access vlan 100

Problem:
ssh'ing into pve (over Mgmt VLAN IP address of .100.12) works. All inbound and outbound communications using default gateway on this port work fine. Ping and communications to/from VLAN 100 via vmbr4 also work fine. But, reachability from vmb4 to other LANs isn't working from pve host itself or from VMs using vmbr4.

Code:
ping 192.168.100.1 -I vmbr4
PING 192.168.100.1 (192.168.100.1) from 192.168.100.41 vmbr4: 56(84) bytes of data.
64 bytes from 192.168.100.1: icmp_seq=1 ttl=64 time=0.130 ms
64 bytes from 192.168.100.1: icmp_seq=2 ttl=64 time=0.050 ms

but ping and communication to/from other VLANs (e.g. 140) via vmb4 do not work.
Code:
ping 192.168.140.1 -I vmbr4
PING 192.168.140.1 (192.168.140.1) from 192.168.100.41 vmbr4: 56(84) bytes of data.
^C
--- 192.168.140.1 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 7145ms

vmbr4 is to be used almost exclusively by VMs on promox. The ping shown above was from pve host itself.

ip r result:
Code:
ip r
default via 192.168.140.1 dev vmbr0 proto kernel onlink
192.168.100.0/24 dev vmbr4 proto kernel scope link src 192.168.100.41
192.168.140.0/24 dev vmbr0 proto kernel scope link src 192.168.140.12

I'm sure its something silly that i'm not seeing.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!