proxmox 1.9 and kernel 2.6.32-6-pve = routing problems with openvz container

valshare · Sep 22, 2011

Hello,

i think there is a bug in the latest kernel. I have upgraded from the kernel 2.18.-6-pve to the latest 2.6.32-6 kernel. With the 2.18.-6 Kernel i can ping from the subnet 192.168.1.0/24 to the subnet 192.168.2.0/24 and backward. I think routing settings are all correct. After install the latest kernel 2.6.32 from pve and pvetest sources the ping fromt one subnet to the other didnt work. Traceroute failed too. After installing the 2.6.18 kernel, all works fine.

Code:

pve-manager: 1.9-24 (pve-manager/1.9/6542)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 1.9-46
pve-kernel-2.6.32-6-pve: 2.6.32-46
qemu-server: 1.1-32
pve-firmware: 1.0-14
libpve-storage-perl: 1.0-19
vncterm: 0.9-2
vzctl: 3.0.29-1pve1
vzdump: 1.2-16
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.0-6

The server01 has follow network config:

Code:

# network interface settings
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
iface eth2 inet manual
iface eth3 inet manual
iface eth4 inet manual
iface eth5 inet manual


auto bond0
iface bond0 inet manual
    slaves eth0 eth1 eth2 eth3
    bond_miimon 100
    bond_mode 802.3ad


auto bond1
iface bond1 inet manual
    slaves eth4
    bond_miimon 100
    bond_mode 802.3ad


auto vmbr0
iface vmbr0 inet static
    address  192.168.1.2
    netmask  255.255.255.0
    gateway  192.168.1.1
    bridge_ports bond0
    bridge_stp on
    bridge_fd 0


auto vmbr1            
iface vmbr1 inet manual
        bridge_ports bond0.2
        bridge_stp off
        bridge_fd 0


auto vmbr10
iface vmbr10 inet manual
    bridge_ports bond0.10
    bridge_stp off
    bridge_fd 0


auto vmbr1000
iface vmbr1000 inet manual
    bridge_ports bond0.1000
    bridge_stp off
    bridge_fd 0


auto vmbr1001
iface vmbr1001 inet manual
    bridge_ports eth5
    bridge_stp off
    bridge_fd 0


auto vmbr6
iface vmbr6 inet manual
    bridge_ports bond1.6
    bridge_stp off
    bridge_fd 0

route tabel:

Code:

Kernel-IP-Routentabelle
Ziel            Router          Genmask         Flags Metric Ref    Use Iface
192.168.1.0     *               255.255.255.0   U     0      0        0 vmbr0
default        192.168.1.1    0.0.0.0         UG    0      0        0 vmbr0

The default gateway 192.168.1.1 nows the route to subnet 192.168.2.0/24 and works with the kernel 2.6.18

Code:

traceroute to 192.168.2.19 (192.168.2.19), 30 hops max, 40 byte packets
 1  192.168.1.1  (192.168.1.1)  1.870 ms  2.438 ms  2.999 ms
 2  192.168.1.11 (192.168.1.11)  3.830 ms  4.309 ms  4.760 ms
 3  192.168.2.19 (192.168.2.19)  5.243 ms  5.695 ms  6.153 ms

There is an old 2.6.18-4 kernel, because of problems with the usb-ip server and kvm. This are the network settings. The routing table is the same as above.

Code:

# network interface settings
auto lo
iface lo inet loopback


iface eth0 inet manual
iface eth1 inet manual


auto bond0
iface bond0 inet manual
    slaves eth0 eth1
    bond_miimon 100
    bond_mode 802.3ad


auto vmbr0
iface vmbr0 inet static
    address  192.168.1.4
    netmask  255.255.255.0
    gateway  192.168.1.1
    bridge_ports bond0
    bridge_stp on
    bridge_fd 0


auto vmbr1
iface vmbr1 inet manual
    bridge_ports bond0.2
    bridge_stp off
    bridge_fd 0

dietmar · Sep 23, 2011

Please try to use tcpdump to see whats wrong.

valshare · Sep 25, 2011

Hello Dietmar,

so, i want to Ping from a KVM Server (192.168.2.10) to a openvz Container with 2 NICs (192.168.1.19 and 192.168.2.19). Both machines are installed on the same Proxmox Server. If i install a 2.6.18 Kernel, all works finde. With 2.6.32.6 i have no chance to reach the destination openvz Interface 192.168.1.19

On the Proxmox Server:

Code:

proxmox01:~# tcpdump -i vmbr1 -vvvv
tcpdump: WARNING: vmbr1: no IPv4 address assigned
tcpdump: listening on vmbr1, link-type EN10MB (Ethernet), capture size 96 bytes
23:34:08.587107 IP (tos 0x0, ttl 128, id 24371, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:34:08.664258 IP (tos 0x0, ttl 128, id 24372, offset 0, flags [none], proto ICMP (1), length 60) kvm.test.local > openvz.test.local: ICMP echo request, id 512, seq 58112, length 40
23:34:13.583194 IP (tos 0x0, ttl 128, id 24377, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:34:18.585152 IP (tos 0x0, ttl 128, id 24386, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:34:22.336374 IP (tos 0x0, ttl 128, id 24391, offset 0, flags [none], proto ICMP (1), length 60) kvm.test.local > openvz.test.local: ICMP echo request, id 512, seq 58368, length 40
23:34:23.580290 IP (tos 0x0, ttl 128, id 24392, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:34:27.674987 IP (tos 0x0, ttl 128, id 24397, offset 0, flags [none], proto ICMP (1), length 60) kvm.test.local > openvz.test.local: ICMP echo request, id 512, seq 58624, length 40
23:34:28.584218 IP (tos 0x0, ttl 128, id 24398, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:34:33.175946 IP (tos 0x0, ttl 128, id 24403, offset 0, flags [none], proto ICMP (1), length 60) kvm.test.local > openvz.test.local: ICMP echo request, id 512, seq 58880, length 40
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel
proxmox01:~#

I think the tcpdump from the Server locks good? ICMP packages are on the vmbr1 interface.

Inside the openvz.test.local server there i can´t see anything from the ICMP packages.

Code:

opsi:/# tcpdump -i eth1 -vvvvvvvvvv
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
23:43:23.583241 IP (tos 0x0, ttl 128, id 25121, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
[B]23:43:23.696010 arp who-has pdc.test.local tell kvm.test.local[/B]
23:43:28.582264 IP (tos 0x0, ttl 128, id 25127, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:43:33.583224 IP (tos 0x0, ttl 128, id 25133, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:43:38.582385 IP (tos 0x0, ttl 128, id 25139, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:43:43.582260 IP (tos 0x0, ttl 128, id 25145, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
23:43:48.580324 IP (tos 0x0, ttl 128, id 25150, offset 0, flags [none], proto UDP (17), length 116) kvm.test.local.1050 > 255.255.255.255.19540: UDP, length 88
^C
7 packets captured
7 packets received by filter
0 packets dropped by kernel

I can see, that there is a arp request for pdc.test.local but i cant reacht it from inside the openvz container from interface eth1 (192.168.2.19). In the openvz container there is a second network interface eth0 (192.168.1.19).
The routing table looks good:

Code:

opsi:/# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.0     *               255.255.255.0   U     0      0        0 eth1
192.168.1.0     *               255.255.255.0   U     0      0        0 eth0
default         192.168.1.1 0.0.0.0         UG    0      0        0 eth0

The default gateway knows the route to the 192.168.2.0/24 network, too.

Now i installed the 2.6.18 kernel and ping from inside the openvz container the pdc.test.local fromt the eth1 interface and it works.

Hope you can help me.

Regards, valle

dietmar · Sep 26, 2011

Please can you test the latest vzctl package:

http://download.proxmox.com/debian/dists/lenny/pvetest/binary-amd64/vzctl_3.0.29-2pve1_amd64.deb

valshare · Sep 26, 2011

Hi,

the new package didn´t resolv the problem.

Code:

proxmox01:/etc/vz/conf# pveversion -v
pve-manager: 1.9-24 (pve-manager/1.9/6542)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 1.9-46
pve-kernel-2.6.32-6-pve: 2.6.32-46
qemu-server: 1.1-32
pve-firmware: 1.0-14
libpve-storage-perl: 1.0-19
vncterm: 0.9-2
vzctl: 3.0.29-2pve1
vzdump: 1.2-16
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.0-6

dietmar · Sep 26, 2011

So basically you never get an ARP reply? Would you mind to report that bug to bugzilla.openvz.org?

dietmar · Sep 26, 2011

dietmar said:
So basically you never get an ARP reply? Would you mind to report that bug to bugzilla.openvz.org?

Well, ignore that comment. Anyways, report to openvz - maybe kir has some idea.

valshare · Sep 26, 2011

Hi Dietmar,

ok, i will report to openvz. Is there a "public formular" to report that direct to kir?

dietmar · Sep 26, 2011

valshare said:
ok, i will report to openvz. Is there a "public formular" to report that direct to kir?

Simply report the bug - there is no need to assign any person.

valshare · Sep 26, 2011

Hi Dietmar,

i have solved the Problem. Reading in the openvz forum gives me the right sign ;-)

Added to /etc/sysctl.d/vzctl.conf the parameter

net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0

dietmar · Sep 27, 2011

valshare said:
Added to /etc/sysctl.d/vzctl.conf the parameter

net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0

Why is that needed? Please can you direct me to the corresponding openvz forum post?

valshare · Sep 27, 2011

dietmar said:
Why is that needed? Please can you direct me to the corresponding openvz forum post?

Yes, of course. Can you Tell me, for what the parameter is? It works, but i dont know for what it is good.

link

dietmar · Sep 29, 2011

valshare said:
Yes, of course. Can you Tell me, for what the parameter is? It works, but i dont know for what it is good.
http://bugzilla.openvz.org/show_bug.cgi?id=1946#c11

No idea. Please can you add a comment to the openvz bug (and tell them that you had the same problem). Maybe they know what that parameter is good for.

dietmar · Sep 29, 2011

Als found this: http://comments.gmane.org/gmane.linux.redhat.release.rhel6/314

Does it work when you set rp_filter to 2?

valshare · Sep 29, 2011

i will try it at the weekend and post the results.

on your postet link, there are the different modes are declared:

rp_filter - INTEGER

0 - No source validation.

1 - Strict mode as defined in RFC3704 Strict Reverse Path
Each incoming packet is tested against the FIB and if the interface
is not the best reverse path the packet check will fail.
By default failed packets are discarded.

2 - Loose mode as defined in RFC3704 Loose Reverse Path
Each incoming packet's source address is also tested against the FIB
and if the source address is not reachable via any interface
the packet check will fail.

Current recommended practice in RFC3704 is to enable strict mode
to prevent IP spoofing from DDos attacks. If using asymmetric routing
or other complicated routing, then loose mode is recommended.

The max value from conf/{all,interface}/rp_filter is used
when doing source validation on the {interface}.

Default value is 0. Note that some distributions enable it
in startup scripts.

dietmar · Sep 29, 2011

And the docs from 2.6.18 say:

rp_filter - BOOLEAN
1 - do source validation by reversed path, as specified in RFC1812
Recommended option for single homed hosts and stub network
routers. Could cause troubles for complicated (not loop free)
networks running a slow unreliable protocol (sort of RIP),
or using static routes.

0 - No source validation.

Search

Search

proxmox 1.9 and kernel 2.6.32-6-pve = routing problems with openvz container

valshare

Active Member

dietmar

Proxmox Staff Member

valshare

Active Member

dietmar

Proxmox Staff Member

valshare

Active Member

dietmar

Proxmox Staff Member

dietmar

Proxmox Staff Member

valshare

Active Member

dietmar

Proxmox Staff Member

valshare

Active Member

dietmar

Proxmox Staff Member

valshare

Active Member

dietmar

Proxmox Staff Member

dietmar

Proxmox Staff Member

valshare

Active Member

dietmar

Proxmox Staff Member