Bridge can ping, vlan can ping, container cannot

S

sadpanda

Active Member
Jul 7, 2020
13
0
41
46
Beating my head against the wall for something that should just work.

Create new linuxBridge on second NIC (vmbr1) click vlan aware, add allowed vlan(s), give it an ip address. Can ping from from router.
assign this bridge to LXC, DHCP and vlan tag, cant ping. Assign w/DHCP, w/o vlan tag, cant ping. Assing with static IP/gateway, cant ping with or w/o vlan tag. Remove IP from bridge, cant ping in any of the above conditions

Create new linuxVLAN, give it an ip address and vlan tag. Can ping that ip from router and can ping it with or w/o ip address applied to vmbr0 bridge AND I can ping it from the LXC! Cant ping router on lxc, cant ping router to LXC, cant select vmbr1.20 vlan as network option for lxc.


My switch is registering the MAC addresses when I apply IP addresses to bridge/vlan and even the lxc so I'm thinking something between lxc and bridge... but firewall is disabled on all interfaces.
 
just gonna start putting the iterations ive tried up here...

the standard vlan aware bridge, vlan tag in LXC
Code: 
auto enp5s0v2
iface enp5s0v2 inet manual

auto enp5s0v3
iface enp5s0v3 inet manual

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp5s0v2
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        
net0: name=eth0,bridge=vmbr1,hwaddr=BC:24:11:1C:CC:02,ip=dhcp,ip6=dhcp,tag=21,type=veth

# nano /etc/systemd/network/eth0.network
[Match]
Name = eth0

[Network]
Description = Interface eth0 autoconfigured by PVE
DHCP = yes
IPv6AcceptRA = false

# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:1c:cc:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::be24:11ff:fe1c:cc02/64 scope link
       valid_lft forever preferred_lft forever
 
vlan on interface, bridge on vlan, bridge NOT vlan aware, LXC vlan not set
Code: 
auto enp5s0v2
iface enp5s0v2 inet manual

auto enp5s0v2.25
iface enp5s0v2.25 inet manual

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp5s0v2.25
        bridge-stp off
        bridge-fd 0

net0: name=eth0,bridge=vmbr1,hwaddr=BC:24:11:1C:CC:02,ip=dhcp,ip6=dhcp,type=veth

# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:1c:cc:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::be24:11ff:fe1c:cc02/64 scope link
       valid_lft forever preferred_lft forever
 
vlan on interface, bridge on vlan, bridge vlan aware, lxc vlan set
Code: 
auto enp5s0v2
iface enp5s0v2 inet manual

auto enp5s0v2.25
iface enp5s0v2.25 inet manual

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp5s0v2.25
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        
net0: name=eth0,bridge=vmbr1,hwaddr=BC:24:11:1C:CC:02,ip=dhcp,ip6=dhcp,tag=25,type=veth
 
Going back to simple bridge/enable vlan and specify tag on LXC...

my switch registers the mac and shows it in the assigned vlan but no dhcp and no traffi

if I tcp dump the nic port I see
Code: 
09:20:47.221613 LLDP, length 271: ArubaS2500-24P-US
09:20:47.394440 STP 802.1s, Rapid STP, CIST Flags [Learn, Forward, Agreement], length 102
09:20:48.470126 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from bc:24:11:1c:cc:02 (oui Unknown), length 294
09:20:48.470579 ARP, Request who-has 172.16.25.101 tell 172.16.25.1, length 42
09:37:55.872313 ARP, Request who-has suchcrap.local.lan tell 172.16.25.1, length 42


if I tcp dump the bridge, I see all the same traffic EXCEPT for the LLDP


I also did some poking regarding my hardware because clearly this should just work... Mellanox Connextx-3.

Code: 
[CODE]# lspci
05:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]
05:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
05:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
05:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
05:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]

# for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
IOMMU group 16 05:00.0 Ethernet controller [0200]: Mellanox Technologies MT27500 Family [ConnectX-3] [15b3:1003]
IOMMU group 17 05:00.1 Ethernet controller [0200]: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] [15b3:1004]
IOMMU group 18 05:00.2 Ethernet controller [0200]: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] [15b3:1004]
IOMMU group 19 05:00.3 Ethernet controller [0200]: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] [15b3:1004]

05:00.1 Is passed to a vm, 'all functions' not checked - all works well. if memory serves, when enabling ethernet mode, the card sets up some virtual nics by default...
remaining resources/second port are presented in networking tab as such:

Code: 
  enp5s0
enp5s0d1
enp5s0v1
enp5s0v2
enp5s0v3

kinda confusing so I assigned bridges and IP addresses (on my mgmt vlan10 same as prox/vmbr0) to each of these interfaces/rebooted and set switch trunkport native vlan to sam (vlan10)

Code: 
6: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP mode DEFAULT group default qlen 1000
    link/ether 00:02:c9:ef:cb:20 brd ff:ff:ff:ff:ff:ff
    vf 0     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
    vf 1     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
    vf 2     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
    vf 3     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
7: enp5s0d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr2 state UP mode DEFAULT group default qlen 1000
    link/ether 00:02:c9:ef:cb:21 brd ff:ff:ff:ff:ff:ff
    vf 0     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
    vf 1     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
    vf 2     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
    vf 3     link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, vlan 4095, spoof checking off, link-state auto
9: enp5s0v1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr3 state UP mode DEFAULT group default qlen 1000
    link/ether 9e:ec:cb:04:72:51 brd ff:ff:ff:ff:ff:ff
10: enp5s0v2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr4 state UP mode DEFAULT group default qlen 1000
    link/ether ba:33:a7:5a:57:c5 brd ff:ff:ff:ff:ff:ff
11: enp5s0v3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr5 state UP mode DEFAULT group default qlen 1000
    link/ether 52:a4:19:37:49:66 brd ff:ff:ff:ff:ff:ff
12: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 3c:52:82:5d:3a:26 brd ff:ff:ff:ff:ff:ff
13: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:02:c9:ef:cb:20 brd ff:ff:ff:ff:ff:ff
14: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:02:c9:ef:cb:21 brd ff:ff:ff:ff:ff:ff
15: vmbr3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 9e:ec:cb:04:72:51 brd ff:ff:ff:ff:ff:ff
16: vmbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether ba:33:a7:5a:57:c5 brd ff:ff:ff:ff:ff:ff
17: vmbr5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 52:a4:19:37:49:66 brd ff:ff:ff:ff:ff:ff

or condensed with ip address assigned
Code: 
  enp5s0 - vmbr1 - .200 - 00:02:c9:ef:cb:20     
enp5s0d1 - vmbr2 - .201 - 00:02:c9:ef:cb:21
enp5s0v1 - vmbr3 - .202 - 9e:ec:cb:04:72:51
enp5s0v2 - vmbr4 - .203 - ba:33:a7:5a:57:c5
enp5s0v3 - vmbr5 - .204 - 52:a4:19:37:49:66

All IPs were pingable

looking at my switch, only MAC addresses for enp5s0d1, enp5s0v2 and enp5s0v3 registered, and from my router ARP table, all IP addresses were present but all on enp5s0d1 MAC

As such I've been testing using enp5s0d1 but so far no joy.
 
You must log in or register to reply here.
Top Bottom