Kernel 2.6.32-4 changes venet behaviour with VLANs -- Problem!

W

wrichter

New Member
Jan 7, 2010
8
0
1
I tried to search the forum for hints but was unsuccessful so far...

I have a PVE HN set up with a single NIC using two VLANs. On VLAN1, the IP subnet 192.168.1.0/24 is configured, on VLAN10 the IP subnet 192.168.10.0/24. A router routes between these subnets.

On Kernel 2.6.24-8, a VE using venet0 with an IP in the 192.168.10.0/24 subnet can successfully ping another system the 192.168.1.0/24 subnet. On Kernel 2.6.32-4, this is no longer possible.

I notice differences in the tcpdumps taken on the HN depending on which kernel it is running. On both kernels, IP packets are sent on the VLAN1 and received on the VLAN10 (I was unsuccessful tweaking the VE_ROUTE_SRC_DEV and NEIGHBOUR_DEVS parameters to only use VLAN10). On 2.6.24-8, the eth0 interface shows only VLAN1 packets, on 2.6.32-4 the eth0 interface shows both VLAN1 and VLAN10 packets. On 2.6.32-4 the venet0 interface shows only outgoing packets, on 2.6.24-8 it shows both incoming and outgoing.

This is the network configuration:
Code: 
root@proxmox:~# cat /etc/network/interfaces 
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
    address  192.168.1.4
    netmask  255.255.255.0
    gateway  192.168.1.1
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

auto vmbr1
iface vmbr1 inet static
    address  192.168.10.3
    netmask  255.255.255.0
    bridge_ports eth0.10
    bridge_stp off
    bridge_fd 0

root@proxmox:~# ifconfig
eth0      Link encap:Ethernet  Hardware Adresse 00:e0:4d:ba:c3:ee  
          inet6-Adresse: fe80::2e0:4dff:feba:c3ee/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metrik:1
          RX packets:15235 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9650 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:1000 
          RX bytes:10426250 (9.9 MiB)  TX bytes:3690756 (3.5 MiB)
          Interrupt:253 Basisadresse:0xe000 

eth0.10   Link encap:Ethernet  Hardware Adresse 00:e0:4d:ba:c3:ee  
          inet6-Adresse: fe80::2e0:4dff:feba:c3ee/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metrik:1
          RX packets:12156 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5240 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0 
          RX bytes:8754714 (8.3 MiB)  TX bytes:3374889 (3.2 MiB)

lo        Link encap:Lokale Schleife  
          inet Adresse:127.0.0.1  Maske:255.0.0.0
          inet6-Adresse: ::1/128 Gültigkeitsbereich:Maschine
          UP LOOPBACK RUNNING  MTU:16436  Metrik:1
          RX packets:3027 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3027 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0 
          RX bytes:1555108 (1.4 MiB)  TX bytes:1555108 (1.4 MiB)

venet0    Link encap:UNSPEC  Hardware Adresse 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          UP BROADCAST PUNKTZUPUNKT RUNNING NOARP  MTU:1500  Metrik:1
          RX packets:3580 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5455 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0 
          RX bytes:199702 (195.0 KiB)  TX bytes:8050495 (7.6 MiB)

vmbr0     Link encap:Ethernet  Hardware Adresse 00:e0:4d:ba:c3:ee  
          inet Adresse:192.168.1.4  Bcast:192.168.1.255  Maske:255.255.255.0
          inet6-Adresse: fe80::2e0:4dff:feba:c3ee/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metrik:1
          RX packets:2093 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4404 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0 
          RX bytes:1412620 (1.3 MiB)  TX bytes:314059 (306.6 KiB)

vmbr1     Link encap:Ethernet  Hardware Adresse 00:e0:4d:ba:c3:ee  
          inet Adresse:192.168.10.3  Bcast:192.168.10.255  Maske:255.255.255.0
          inet6-Adresse: fe80::2e0:4dff:feba:c3ee/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metrik:1
          RX packets:11780 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4502 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0 
          RX bytes:8723716 (8.3 MiB)  TX bytes:2727969 (2.6 MiB)
TCPdumps in the next post due to size limit...
 
Last edited:
TCPdumps of a ping request on 2.6.24-8:
Code: 
root@proxmox:~# uname -a
Linux proxmox 2.6.24-8-pve #1 SMP PREEMPT Fri Oct 16 11:17:55 CEST 2009 x86_64 GNU/Linux
root@proxmox:~# tcpdump -n -i eth0 host 192.168.10.20
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
01:28:57.475911 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 40449, seq 1, length 64
01:28:58.474921 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 40449, seq 2, length 64
01:28:59.473922 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 40449, seq 3, length 64

3 packets captured
4 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i eth0.10 host 192.168.10.20
tcpdump: WARNING: eth0.10: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0.10, link-type EN10MB (Ethernet), capture size 96 bytes
01:29:10.724023 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 40705, seq 1, length 64
01:29:11.722909 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 40705, seq 2, length 64
01:29:12.721935 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 40705, seq 3, length 64

3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i venet0 host 192.168.10.20
tcpdump: WARNING: arptype 65535 not supported by libpcap - falling back to cooked socket
tcpdump: WARNING: venet0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on venet0, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
01:29:31.724166 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 40961, seq 1, length 64
01:29:31.724563 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 40961, seq 1, length 64
01:29:32.723163 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 40961, seq 2, length 64
01:29:32.723442 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 40961, seq 2, length 64
01:29:33.722162 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 40961, seq 3, length 64
01:29:33.722407 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 40961, seq 3, length 64

6 packets captured
6 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i vmbr0 host 192.168.10.20
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr0, link-type EN10MB (Ethernet), capture size 96 bytes
01:30:06.649755 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 41217, seq 1, length 64
01:30:07.649618 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 41217, seq 2, length 64
01:30:08.649616 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 41217, seq 3, length 64

3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i vmbr1 host 192.168.10.20
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 96 bytes
01:30:19.413091 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 41473, seq 1, length 64
01:30:20.411969 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 41473, seq 2, length 64
01:30:21.410942 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 41473, seq 3, length 64

3 packets captured
3 packets received by filter
0 packets dropped by kernel
TCPdumps of a ping request on 2.6.32-4:
Code: 
root@proxmox:~# uname -a
Linux proxmox 2.6.32-4-pve #1 SMP Wed Dec 15 14:04:31 CET 2010 x86_64 GNU/Linux
root@proxmox:~# tcpdump -n -i eth0 host 192.168.10.20
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
01:39:00.612631 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 38913, seq 1, length 64
01:39:00.612984 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 38913, seq 1, length 64
01:39:01.620671 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 38913, seq 2, length 64
01:39:01.620930 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 38913, seq 2, length 64
01:39:02.628661 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 38913, seq 3, length 64
01:39:02.628883 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 38913, seq 3, length 64
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i eth0.10 host 192.168.10.20
tcpdump: WARNING: eth0.10: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0.10, link-type EN10MB (Ethernet), capture size 96 bytes
01:39:17.265636 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 39169, seq 1, length 64
01:39:18.272914 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 39169, seq 2, length 64
01:39:19.280872 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 39169, seq 3, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i venet0 host 192.168.10.20
tcpdump: WARNING: arptype 65535 not supported by libpcap - falling back to cooked socket
tcpdump: WARNING: venet0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on venet0, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
01:39:50.501274 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 39425, seq 1, length 64
01:39:51.508652 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 39425, seq 2, length 64
01:39:52.516628 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 39425, seq 3, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i vmbr0 host 192.168.10.20
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr0, link-type EN10MB (Ethernet), capture size 96 bytes
01:40:15.553288 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 39681, seq 1, length 64
01:40:16.560670 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 39681, seq 2, length 64
01:40:17.568649 IP 192.168.10.20 > 192.168.1.1: ICMP echo request, id 39681, seq 3, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@proxmox:~# tcpdump -n -i vmbr1 host 192.168.10.20
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 96 bytes
01:40:37.021625 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 39937, seq 1, length 64
01:40:38.028908 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 39937, seq 2, length 64
01:40:39.036881 IP 192.168.1.1 > 192.168.10.20: ICMP echo reply, id 39937, seq 3, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
 
wrichter said:
I have a PVE HN set up with a single NIC using two VLANs. On VLAN1, the IP subnet 192.168.1.0/24 is configured, on VLAN10 the IP subnet 192.168.10.0/24. A router routes between these subnets.
Click to expand...

Your network configuration shows only one vlan?
 
dietmar said:
Your network configuration shows only one vlan?
Click to expand...

/etc/network/interfaces only has a stanza for eth0, however the vmbr1 stanza directly references eth0.10. ifconfig shows both eth0 and eth0.10 configured. This was the configuration as set up through the Proxmox Web UI, I didn't change anything manually.

The two vmbr devices do work correctly under both kernels, i.e. veth devices connected to vmbr0 talk on VLAN1 and veth devices connected to vmbr1 talk on VLAN10.
 
Hi,
but that's not correct. You must enable vlan-tagging for all used vlans. Mixed with default-vlan and tagged vlan don't work.
If you enable vlan-tagging for vlan1 and use eth0.1 it should work.

Udo
 
udo said:
Hi,
but that's not correct. You must enable vlan-tagging for all used vlans. Mixed with default-vlan and tagged vlan don't work.
If you enable vlan-tagging for vlan1 and use eth0.1 it should work.

Udo
Click to expand...

Sorry for being dense, but I don't really know what to do here. I can't get the venet0 to work properly (well, unless I switch back to the older kernel, then the behaviour may not be correct, but at least it'll work).

Things I have tried (everything on the 2.6.32-4 kernel):

  • removed the eth0 stanza from /etc/network/interfaces
  • add stanzas for eth0.1 and eth0.1 similar to the eth0 stanza
  • reference eth0.1 instead of eth0 in the vmbr0 stanza in /etc/network/interfaces
  • changed the VLAN switch not assign untagged packets to VLAN1 but to only accept tagged VLAN1 packets
My /etc/network/interfaces now looks as follows:
Code: 
root@proxmox:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

#iface eth0 inet manual
iface eth0.1 inet manual
iface eth0.10 inet manual

auto vmbr0
iface vmbr0 inet static
    address  192.168.1.4
    netmask  255.255.255.0
    gateway  192.168.1.1
    bridge_ports eth0.1
    bridge_stp off
    bridge_fd 0

auto vmbr1
iface vmbr1 inet static
    address  192.168.10.3
    netmask  255.255.255.0
    bridge_ports eth0.10
    bridge_stp off
    bridge_fd 0
TCPdumps on the eth0[.1|.10] interfaces show that the outgoing ICMP packets are still routed (incorrectly) to the VLAN1 interface and replys are (correctly) received on the VLAN10 interface. All packets turn up when tracing the default eth0 adapter.

Code: 
root@proxmox:~# tcpdump -n -i eth0.10 host 192.168.10.6
tcpdump: WARNING: eth0.10: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0.10, link-type EN10MB (Ethernet), capture size 96 bytes
12:23:20.408970 IP 192.168.1.1 > 192.168.10.6: ICMP echo reply, id 40705, seq 1, length 64
12:23:21.416330 IP 192.168.1.1 > 192.168.10.6: ICMP echo reply, id 40705, seq 2, length 64
12:23:22.424348 IP 192.168.1.1 > 192.168.10.6: ICMP echo reply, id 40705, seq 3, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

root@proxmox:~# tcpdump -n -i eth0.1 host 192.168.10.6
tcpdump: WARNING: eth0.1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0.1, link-type EN10MB (Ethernet), capture size 96 bytes
12:23:36.016677 IP 192.168.10.6 > 192.168.1.1: ICMP echo request, id 40961, seq 1, length 64
12:23:37.024122 IP 192.168.10.6 > 192.168.1.1: ICMP echo request, id 40961, seq 2, length 64
12:23:38.032107 IP 192.168.10.6 > 192.168.1.1: ICMP echo request, id 40961, seq 3, length 64
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

root@proxmox:~# tcpdump -n -i eth0 host 192.168.10.6
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
12:29:08.352802 IP 192.168.10.6 > 192.168.1.1: ICMP echo request, id 42497, seq 1, length 64
12:29:08.353186 IP 192.168.1.1 > 192.168.10.6: ICMP echo reply, id 42497, seq 1, length 64
12:29:09.360125 IP 192.168.10.6 > 192.168.1.1: ICMP echo request, id 42497, seq 2, length 64
12:29:09.360356 IP 192.168.1.1 > 192.168.10.6: ICMP echo reply, id 42497, seq 2, length 64
12:29:10.368106 IP 192.168.10.6 > 192.168.1.1: ICMP echo request, id 42497, seq 3, length 64
12:29:10.368332 IP 192.168.1.1 > 192.168.10.6: ICMP echo reply, id 42497, seq 3, length 64
My routing table does look correct however:
Code: 
root@proxmox:~# route -n
Kernel-IP-Routentabelle
Ziel            Router          Genmask         Flags Metric Ref    Use Iface
192.168.10.6    0.0.0.0         255.255.255.255 UH    0      0        0 venet0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 vmbr0
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 vmbr1
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 vmbr0
Interestingly, everything is routed correctly when pinging directly from the HN.
 
You must log in or register to reply here.
Top Bottom