802.1ad QinQ issue

jarekm95

New Member
Jul 19, 2019
1
0
1
29
Hello.
I have problem with double vlan tagging setup. I want to split 'trunk' link to different VMs using 802.1ad tag, and split 802.1q tags on the VM (on the guest machines).
My /etc/network/interfaces file:
Code:
auto lo
iface lo inet loopback

iface enp0s3 inet manual

auto enp0s8
iface enp0s8 inet manual
        post-up ip link add link enp0s8 enp0s8.104 type vlan proto 802.1ad id 104
        post-up ip link set enp0s8.104 up

auto enp0s8.104
iface enp0s8.104 inet manual

auto vmbr0
iface vmbr0 inet static
        address  192.168.0.200
        netmask  255.255.255.0
        gateway  192.168.0.1
        bridge-ports enp0s3
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address  192.168.104.200
        netmask  255.255.255.0
        bridge-ports enp0s8.104
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

Network interfaces looks good:
Code:
root@pve:~# ip -d link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:38:51:9f brd ff:ff:ff:ff:ff:ff promiscuity 1
    bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0 designated_bridge 8000.8:0:27:38:51:9f designated_root 8000.8:0:27:38:51:9f hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off group_fwd_mask 0x0 group_fwd_mask_str 0x0 vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:5d:3c:ac brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
4: enp0s8.104@enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr1 state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:5d:3c:ac brd ff:ff:ff:ff:ff:ff promiscuity 0
    vlan protocol 802.1ad id 104 <REORDER_HDR>
    bridge_slave state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0 designated_bridge 8000.8:0:27:5d:3c:ac designated_root 8000.8:0:27:5d:3c:ac hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on neigh_suppress off group_fwd_mask 0x0 group_fwd_mask_str 0x0 vlan_tunnel off addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
5: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:5d:3c:ac brd ff:ff:ff:ff:ff:ff promiscuity 0
    bridge forward_delay 0 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32768 vlan_filtering 1 vlan_protocol 802.1Q bridge_id 8000.8:0:27:5d:3c:ac designated_root 8000.8:0:27:5d:3c:ac root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer    0.00 tcn_timer    0.00 topology_change_timer    0.00 gc_timer  259.76 vlan_default_pvid 1 vlan_stats_enabled 0 group_fwd_mask 0 group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 4 mcast_hash_max 512 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3124 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:38:51:9f brd ff:ff:ff:ff:ff:ff promiscuity 0
    bridge forward_delay 0 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id 8000.8:0:27:38:51:9f designated_root 8000.8:0:27:38:51:9f root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer    0.00 tcn_timer    0.00 topology_change_timer    0.00 gc_timer  251.19 vlan_default_pvid 1 vlan_stats_enabled 0 group_fwd_mask 0 group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 4 mcast_hash_max 512 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3124 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

But when i trying to ping 192.168.104.200 (or attached to bridge VPC), I see only ARP, no ICMP:
Code:
root@pve:~# tcpdump -i vmbr1 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:26:02.960792 9a:f3:e7:e5:64:e5 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 56: Request who-has 192.168.104.200 tell 192.168.104.1, length 42
21:26:02.960812 08:00:27:5d:3c:ac (oui Unknown) > 9a:f3:e7:e5:64:e5 (oui Unknown), ethertype ARP (0x0806), length 42: Reply 192.168.104.200 is-at 08:00:27:5d:3c:ac (oui Unknown), length 28
21:26:04.010685 9a:f3:e7:e5:64:e5 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 56: Request who-has 192.168.104.200 tell 192.168.104.1, length 42
21:26:04.010702 08:00:27:5d:3c:ac (oui Unknown) > 9a:f3:e7:e5:64:e5 (oui Unknown), ethertype ARP (0x0806), length 42: Reply 192.168.104.200 is-at 08:00:27:5d:3c:ac (oui Unknown), length 28
21:26:05.041341 9a:f3:e7:e5:64:e5 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 56: Request who-has 192.168.104.200 tell 192.168.104.1, length 42
21:26:05.041371 08:00:27:5d:3c:ac (oui Unknown) > 9a:f3:e7:e5:64:e5 (oui Unknown), ethertype ARP (0x0806), length 42: Reply 192.168.104.200 is-at 08:00:27:5d:3c:ac (oui Unknown), length 28
21:26:06.081499 9a:f3:e7:e5:64:e5 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 56: Request who-has 192.168.104.200 tell 192.168.104.1, length 42
21:26:06.081528 08:00:27:5d:3c:ac (oui Unknown) > 9a:f3:e7:e5:64:e5 (oui Unknown), ethertype ARP (0x0806), length 42: Reply 192.168.104.200 is-at 08:00:27:5d:3c:ac (oui Unknown), length 28
^C
8 packets captured
18 packets received by filter
0 packets dropped by kernel
root@pve:~# tcpdump -i enp0s8 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes
21:26:53.930819 9a:f3:e7:e5:64:e5 (oui Unknown) > Broadcast, ethertype 802.1Q-QinQ (0x88a8), length 60: vlan 104, p 0, ethertype ARP, Request who-has 192.168.104.200 tell 192.168.104.1, length 42
21:26:53.930869 08:00:27:5d:3c:ac (oui Unknown) > 9a:f3:e7:e5:64:e5 (oui Unknown), ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 104, p 0, ethertype ARP, Reply 192.168.104.200 is-at 08:00:27:5d:3c:ac (oui Unknown), length 28
21:26:54.961025 9a:f3:e7:e5:64:e5 (oui Unknown) > Broadcast, ethertype 802.1Q-QinQ (0x88a8), length 60: vlan 104, p 0, ethertype ARP, Request who-has 192.168.104.200 tell 192.168.104.1, length 42
21:26:54.961076 08:00:27:5d:3c:ac (oui Unknown) > 9a:f3:e7:e5:64:e5 (oui Unknown), ethertype 802.1Q-QinQ (0x88a8), length 46: vlan 104, p 0, ethertype ARP, Reply 192.168.104.200 is-at 08:00:27:5d:3c:ac (oui Unknown), length 28
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel
root@pve:~#

I disabled firewall on GUI. This config works when I replaced proxmox host by host with raspbian OS.
Why this don't work here?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!