CT using 2 IP going to different network (venet routing problem)

[stef1777]

Hello!

I've a setup a config using multiple gateway as in this web page. All is working fine on the PVE node.

http://generationip.com/documentati...howto-setup-multiple-default-gateway-on-linux

On top of this, I mounted a CT with 2 IPs (with venet). One on the left network, one on the right network.

The left one (vmbr0) is working fine. I can ping it. The second at the right (vmbr1) is not working.

I'll looking for a solution to have the CT answering on both network. I tried to define route for the CT but wthout success.

Bellow is the config on the PVE node.


# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

iface eth2 inet manual

iface eth3 inet manual

auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
bond_mode active-backup

auto bond1
iface bond1 inet manual
slaves eth2 eth3
bond_miimon 100
bond_mode active-backup

auto vmbr0
iface vmbr0 inet static
address 62.xxx.212.17
netmask 255.255.255.0
gateway 62.xxx.212.254
bridge_ports bond0
bridge_stp off
bridge_fd 0
# Edit /etc/iproute2/rt_tables and add table t17
post-up ip route add 62.xxx.212.0/24 dev vmbr0 src 62.xxx.212.17 table t17
post-up ip route add default via 62.xxx.212.254 dev vmbr0 src 62.xxx.212.17 table t17
post-up ip rule add from 62.xxx.212.17 table t17
post-up ip route add 127.0.0.0/8 dev lo table t17
post-up ip route add 195.yyy.139.96/27 dev vmbr1 table t17

auto vmbr1
iface vmbr1 inet static
address 195.yyy.139.123
netmask 255.255.255.224
bridge_ports bond1
bridge_stp off
bridge_fd 0
# Edit /etc/iproute2/rt_tables and add table t123
post-up ip route add 195.yyy.139.96/27 dev vmbr1 src 195.yyy.139.123 table t123
post-up ip route add default via 195.yyy.139.126 dev vmbr1 src 195.yyy.139.123 table t123
post-up ip rule add from 195.yyy.139.123 table t123
post-up ip route add 127.0.0.0/8 dev lo table t123
post-up ip route add 62.xxx.212.0/24 dev vmbr0 table t123

 

[m.ardito]

did you try do put a (right) gateway line to vmbr1?

Marco

 

[stef1777]

Hello!

In fact I solved part of the problem adding theses ip route to vmbr1.

# Edit /etc/iproute2/rt_tables and add table t99
post-up ip rule add from 195.yyy.139.99 table t99
post-up ip route add default dev vmbr1 via 195.yyy.139.126 table t99
post-up ip route add 195.yyy.139.96/27 dev vmbr1 table t99
post-up ip route add table t99 "195.yyy.139.99" dev venet0 scope link

I can now ping the right CT server IP (195.yyy.139.99).

But all is not perfert. Some route seems still missing.

From inside the server, theses pings do not works.

ping -I lo 195.yyy.139.99 OK
ping -I lo 62.xxx.212.120 OK
ping -I vmbr0 195.yyy.139.99 NO
ping -I vmbr1 195.yyy.139.99 NO
ping -I vmbr0 62.xxx.212.120 NO
ping -I vmbr1 62.xxx.212.120 NO
ping 195.yyy.139.99 NO
ping 62.xxx.212.120 NO


I also have this problem from the net.

[MacMiam:~] sxs% traceroute -n -I 195.yyy.139.99
traceroute to 195.yyy.139.99 (195.yyy.139.99), 64 hops max, 72 byte packets
1 192.168.0.254 0.405 ms 0.428 ms 0.198 ms
2 82.xx.159.254 36.407 ms 37.850 ms 33.020 ms
3 213.xxx.4.254 38.927 ms 38.606 ms 38.672 ms
4 212.xx.56.149 40.899 ms 27.853 ms 40.055 ms
5 212.xx.59.101 37.215 ms 38.209 ms 58.410 ms
6 212.xx.58.50 34.878 ms 29.215 ms 39.171 ms
7 88.xxx.1.131 39.939 ms 39.492 ms 38.553 ms
8 195.xxx.1.17 37.733 ms 38.606 ms 38.413 ms
9 195.1xxx.1.25 41.359 ms 39.656 ms 35.763 ms
10 * * *
11 195.yyy.139.99 36.759 ms 38.956 ms 38.290 ms

Hops 10 is missing. Should be the server 195.yyy.139.123 ?

Suggestion from reader welcome ! ;-)

 

[stef1777]

Hello!

The double gateway CT is finaly working. Thanks to multiple gateway routing.

Below is the final version. Working fine. Some trouble with some route with ping but core is working.


auto vmbr0
iface vmbr0 inet static
address 62.xxx.212.17
netmask 255.255.255.0
gateway 62.xxx.212.254
bridge_ports bond0
bridge_stp off
bridge_fd 0
# Edit /etc/iproute2/rt_tables and add table t17
post-up ip route add 62.xxx.212.0/24 dev vmbr0 src 62.xxx.212.17 table t17
post-up ip route add default via 62.xxx.212.254 dev vmbr0 src 62.xxx.212.17 table t17
post-up ip rule add from 62.xxx.212.17 table t17
post-up ip route add 127.0.0.0/8 dev lo table t17
post-up ip route add 195.yyy.139.96/27 dev vmbr1 table t17


auto vmbr1
iface vmbr1 inet static
address 195.yyy.139.123
netmask 255.255.255.224
bridge_ports bond1
bridge_stp off
bridge_fd 0
# Edit /etc/iproute2/rt_tables and add table t123
post-up ip route add 195.yyy.139.96/27 dev vmbr1 src 195.yyy.139.123 table t123
post-up ip route add default via 195.yyy.139.126 dev vmbr1 src 195.yyy.139.123 table t123
post-up ip rule add from 195.yyy.139.123 table t123
post-up ip route add 127.0.0.0/8 dev lo table t123
post-up ip route add 62.xxx.212.0/24 dev vmbr0 table t123
post-up ip rule add from 195.yyy.139.99 table t123
post-up ip route add default dev vmbr1 via 195.yyy.139.126 table t123
post-up ip route add 195.yyy.139.96/27 dev vmbr1 table t123