Policy-based routing in LXC Container

alexc

Renowned Member
Apr 13, 2015
138
4
83
I have LXC container on PVE host, and initially it was assigned one WAN IP and one LAN IP (LAN IP is internal subnet to reach other VMs on PVE host). It was configured via web and this is how it looked in the config:

net0: name=eth0,bridge=vmbr0,firewall=1,gw=xx.xx.195.17,hwaddr=36:69:FC:25:11:CB,ip=xx.xx.195.24/28,type=veth
net1: name=eth1,bridge=vmbr1,firewall=1,hwaddr=CE:F4:78:24:DA:EF,ip=192.168.0.107/24,tag=100,type=veth

this small config worked fine and web gui was perfect for the task.

After a while, the need to add more IPs to the container arised. ISP was unable to make mask bigger, so then assigned anothet /28 subnet to the host, which has different gw.

I did the trick like this: in the container I added extra IPs as extra nets and ssigned new IPs to it, and set up source based routing so traffic from new IPs be pointed to new gateway:

Code:
CT# cat /etc/network/interfaces
auto lo
iface lo inet loopback
    post-up grep -q "^102 eth2-eth5$" /etc/iproute2/rt_tables || echo "102 eth2-eth5" >> /etc/iproute2/rt_tables
    post-up ip route add default via xx.xx.195.241 dev eth2 table 102
    post-up ip rule add from xx.xx.195.242 lookup 102
    post-up ip rule add from xx.xx.195.243 lookup 102
    post-up ip rule add from xx.xx.195.244 lookup 102
    post-up ip rule add from xx.xx.195.245 lookup 102


auto eth0
allow-hotplug eth0
iface eth0 inet static
        address xx.xx.195.24/28
        gateway xx.xx.195.17


auto eth1
allow-hotplug eth1
iface eth1 inet static
        address 192.168.0.107/24

auto eth2
allow-hotplug eth2
iface eth2 inet static
        address xx.xx.195.242/28


auto eth3
allow-hotplug eth3
iface eth3 inet static
        address xx.xx.195.243/28


auto eth4
allow-hotplug eth4
iface eth4 inet static
        address xx.xx.195.244/28

auto eth5
allow-hotplug eth5
iface eth5 inet static
        address xx.xx.195.245/28

This started to work after test reboot but after a while output of ip rule show changed to simple
Code:
# ip rule show
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

I run commands from the post-up lines above and got the output a bit bigger:
Code:
# ip rule show
0:      from all lookup local
32762:  from xx.xx.195.245 lookup eth2-eth5
32763:  from xx.xx.195.244 lookup eth2-eth5
32764:  from xx.xx.195.243 lookup eth2-eth5
32765:  from xx.xx.195.242 lookup eth2-eth5
32766:  from all lookup main
32767:  from all lookup default

So, the questions is:
1. If source-based routing fits the container idea?
2. How to prevents /etc/network/interfaces from overwrite on each boot (or whenever Proxmox decide to reconfigure the container network).

There are rumors about file named /etc/network/.pve-ignore.interfaces but I see no docs for this naming, so seems I need to create files that have .pve-ignore. and then original filename to save files intact. But I also afraid how PVE will treat such LXC container, for Proxmox want to configure containers.

Please advice how to deal in such a situation.

And how can I find how often PVE try to change ip addresses, routes and rules within container?
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!