Network Routing

kirelgt

Member
Nov 16, 2010
85
0
6
Hello everyone. Here is a little problem I am having that is driving me crazy. I have 4 NIC cards in a server. Here is the configuration. There is a routing problem in the system. When I assign two NIC cards to a VM (vmbr0 and vmbr1). When I tried to ping 192.168.1.0 inside the VM, the traffic actually goes through the vmbr0, when it should be going to vmbr1. Please advise.


auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 1000
bond_mode active-backup


auto bond1
iface bond1 inet manual
slaves eth2 eth3
bond_miimon 1000
bond_mode active-backup


auto vmbr0
iface vmbr0 inet static
address 10.10.10.40
netmask 255.255.255.0
gateway 10.10.10.1
bridge_ports bond0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 192.168.1.40
netmask 255.255.255.0
gateway 192.168.1.40
bridge_ports bond1
bridge_stp off
bridge_fd 0
 
Hello everyone. Here is a little problem I am having that is driving me crazy. I have 4 NIC cards in a server. Here is the configuration. There is a routing problem in the system. When I assign two NIC cards to a VM (vmbr0 and vmbr1). When I tried to ping 192.168.1.0 inside the VM, the traffic actually goes through the vmbr0, when it should be going to vmbr1. Please advise.
...

Hi,
perhaps the nic-order inside the VM changed? Compare the mac-addresses, and perhaps create an udev-rule inside the VM.

Udo
 
Udo thank you for your time. The weird thing is that if I even ping from the proxmox itself it does not ping. I have to tell you that this only happens with vmbr1. For instance if I ping on the 10.x.x.x network I can ping from proxmox and the VM. Now if I try to do the same with the 192.x.x.x network I cannot ping from proxmox or the VM. It is weird.
 
Udo thank you for your time. The weird thing is that if I even ping from the proxmox itself it does not ping. I have to tell you that this only happens with vmbr1. For instance if I ping on the 10.x.x.x network I can ping from proxmox and the VM. Now if I try to do the same with the 192.x.x.x network I cannot ping from proxmox or the VM. It is weird.
Hi kirelgt,
do you ping from the host to 192.x.x.x or to 192.168.1.x?
I ask because of your gateway-entry on vmbr1 - this entry make no sense - please remove (you should allways have only one default gateway).
Look with "ip route" - i assume that you must delete one default-entry (ip route del default via 192.168.1.40).

Are both interfaces from vmbr1 on the switch in the right vlan? Does the same happens with only one interface on the bond?

Udo
 
Well Udo the 192.x.x.x is in reality 192.168.1.x; it is the only 192 network. The switch I know it is not the problem and I explain you why. I open two ssh connection to the server and I do a tcpdump in one of them and the other I start a ping in the 192 network. When I go to the dump I can see that the ping is going out the wrong interface.

Here is a #ip route
192.168.1.0/24 dev vmbr1 proto kernel scope link src 192.168.1.40
10.10.10.0/24 dev vmbr0 proto kernel scope link src 10.10.10.40
default via 10.10.10.1 dev vmbr0



Here is a #route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr1
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr0
0.0.0.0 10.10.10.1 0.0.0.0 UG 0 0 0 vmbr0

As you can see everything is perfect but still route the wrong way. Can you see anything configure incorrectly.

Note: Sorry about the gateway part in the vmbr1, there is none actually there.
 
Last edited:
Well Udo the 192.x.x.x is in reality 192.168.1.x; it is the only 192 network. The switch I know it is not the problem and I explain you why. I open two ssh connection to the server and I do a tcpdump in one of them and the other I start a ping in the 192 network. When I go to the dump I can see that the ping is going out the wrong interface.

Here is a #ip route
192.168.1.0/24 dev vmbr1 proto kernel scope link src 192.168.1.40
10.10.10.0/24 dev vmbr0 proto kernel scope link src 10.10.10.40
default via 10.10.10.1 dev vmbr0



Here is a #route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr1
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 vmbr0
0.0.0.0 10.10.10.1 0.0.0.0 UG 0 0 0 vmbr0

As you can see everything is perfect but still route the wrong way. Can you see anything configure incorrectly.

Note: Sorry about the gateway part in the vmbr1, there is none actually there.
Hi kirelgt,
there is all configured right - there must be an mistake in port order if on the switch side is all ok.
Have you checked /etc/udev/rules.d/70-persistent-net.rules and/or "dmesg | grep eth" perhaps the port naming changed from udev?
If you do an "tcpdump -i eth0" (or eth1) do you see any traffic 192.168.1-related? Or on eth2/3 10.10.10-related?

Udo
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!