vmbr0 Loosing Inbound Traffic

MeCJay12

New Member
Feb 19, 2023
12
0
1
Hello! New to Proxmox and forums (experienced in other HA hypervisors). I just setup a 2-node cluster and I've been having some networking issues. I have two interfaces (enp1s0f1 primary & enp1s0f0 backup) in an active-passive bond (bond0). I then setup vmbr0 on bond0 and vlan2 onto vmbr0. Initially, vlan2 didn't work. I didn't think much of it (I've seen similar bugs in Hyper-V), I just moved vlan2 to bond0 and everything was working. Then later, once I had my first VM VM installed, I noticed that it, connected to vmbr0 with vlan tag 2, couldn't connect to the network. This time I dug into it much deeper. I eventually configured a new Test interface with vlan tag 2 on vmbr0 and IP 192.168.2.75 and did some packet captures to find that it looks like vmbr0 is passing traffic out but not accepting it back. I have the firewall turned off on the VM and as far as I can tell, there's no default firewall for the host so I'm not sure what's going on. Thanks in advance for the help!

Pining out from the Test interface on the host itself
Code:
cs@prLabA:~$ ping -I Test 192.168.2.1
PING 192.168.2.1 (192.168.2.1) from 192.168.2.75 Test: 56(84) bytes of data.

ARP request going out the physical interface and the ARP reply coming back
Code:
cs@prLabA:~$ sudo tcpdump host 192.168.2.75 -vvv -i enp1s0f1
tcpdump: listening on enp1s0f1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:33:28.354821 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.2.1 tell 192.168.2.75, length 28
19:33:28.354861 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.2.1 is-at 00:00:5e:00:01:02 (oui IANA), length 46

Same thing on bond0
Code:
cs@prLabA:~$ sudo tcpdump host 192.168.2.75 -vvv -i bond0
tcpdump: listening on bond0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:33:34.498795 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.2.1 tell 192.168.2.75, length 28
19:33:34.498844 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.2.1 is-at 00:00:5e:00:01:02 (oui IANA), length 46

Replies disappear from vmbr0
Code:
cs@prLabA:~$ sudo tcpdump host 192.168.2.75 -vvv -i vmbr0
tcpdump: listening on vmbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:33:41.666722 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.2.1 tell 192.168.2.75, length 28
19:33:42.690747 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.2.1 tell 192.168.2.75, length 28

Configs
Code:
root@prLabA:/home/cshaheen# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp1s0f0
iface enp1s0f0 inet manual

iface eno1 inet manual

iface eno2 inet manual

auto enp1s0f1
iface enp1s0f1 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves enp1s0f0 enp1s0f1
        bond-miimon 100
        bond-mode active-backup
        bond-primary enp1s0f1

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vlan2
iface vlan2 inet static
        address 192.168.2.48/24
        gateway 192.168.2.1
        vlan-raw-device bond0

iface vlan2 inet6 static
        address fc00:0:0:2::48/64

auto Test
iface Test inet static
        address 192.168.2.75/24
        vlan-id 2
        vlan-raw-device vmbr0
 

Attachments

  • vmbr0.png
    vmbr0.png
    16.7 KB · Views: 3
  • Test.png
    Test.png
    26.4 KB · Views: 3
  • vlan2.png
    vlan2.png
    28.5 KB · Views: 2
  • bond0.png
    bond0.png
    19.9 KB · Views: 2
  • enp1s0f1.png
    enp1s0f1.png
    15.3 KB · Views: 2
  • enp1s0f0.png
    enp1s0f0.png
    14.6 KB · Views: 2
  • vm100.png
    vm100.png
    19.9 KB · Views: 3
Last edited:
For anyone that come across this in the future, I switched to OVS and that fixed everything.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!