Permanent routing in Debian 11 LXC container

m3a2r1 · Feb 2, 2023

I need to set permanent routing to Debian 11 container but I can't. It reverts to default on every restart of container.
How to set it properly?

Moayad · Feb 2, 2023

Hi,

In the `/etc/network/interfaces` inside the CT itself. However, by default, Proxmox VE modifies some files at the container startup, and to let the PVE not modifies/edits the files you can touch an empty file with .pve-ignore as prefix. i.e, if you not need the PVE edit the /etc/hosts at the startup, you can do the following command:

Bash:

touch /etc/.pve-ignore.hosts

See the link below [0] for more information.

[0] https://pve.proxmox.com/wiki/Linux_Container#_guest_operating_system_configuration

m3a2r1 · Feb 2, 2023

I've tried to add 'ip route add' to /etc/network/interfaces earlier but it is ignored.

Moayad · Feb 2, 2023

Hi,

That should work, make sure to restart the network service systemctl restart networking after you add a `up route add -net` to the Container network configuration.

m3a2r1 · Feb 2, 2023

I've tested with "ip" not "route" command. I'll try with route, which package is it?

Dunuin · Feb 16, 2024

Same problem here. Got a Debian 12 LXC and I need to set some routes.
Usually I would add a pre-up to the interface in /etc/network/interfaces to run the "ip route add" command. But this isn't great if the interface is edited by PVE to match the LXC config.
Using "/etc/network/.pve-ignore.interfaces" doesn't sound great either with stuff configured in the webUI being fully ignored. I bet some years later I try to edit the network config via webUI and will wonder why the LXC isn't working...

I tried to add the routes via /etc/crontab...

Code:

# Set manual routes after reboot
@reboot root /usr/sbin/ip route add 192.168.4.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.41.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.42.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.44.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.46.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.47.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.50.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1

...but this isn't reliable and will sometimes add the routes, sometime not, after a LXC reboot.

Only other thing I could think of is adding a oneshot systemd service. I guess I would need to add a "After=network-online.target" there so adding the routes won't fail because interfaces aren't initialized yet?

Using systemd-networkd also isn't an option as I needed to disable it when upgrading that LXC from Debian 11 to 12 because of the "Failed to start systemd-networkd-wait-online.service – Wait for Network to be Configured." error.

So what's the best way to use manual routes in an LXC?

_gabriel · Feb 16, 2024

perhaps a /etc/network/if-up.d/custom-routes.sh ?

Dunuin · Feb 16, 2024

Thanks. Sounds good. Will try that and report back.

Dunuin · Feb 25, 2024

I created such a script for the PVE hosts and there it is working fine when using "[ "$IFACE" = "vmbr43" ]". But when I use the same inside but changing it to "eth0" the Debian LXC it isn't adding routes.
Some data:

This in the LXCs crontab is sometimes working, sometimes not:

Code:

# Set static routes after reboot
@reboot root /usr/sbin/ip route add 192.168.4.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.41.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.42.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.44.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.46.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.47.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1
@reboot root /usr/sbin/ip route add 192.168.50.0/24 via 192.168.43.1 dev eth0 > /dev/null 2>&1

I removed the cron and replaced it with these:

Code:

touch /etc/network/if-up.d/custom-routes.sh
chown root:root /etc/network/if-up.d/custom-routes.sh
chmod 750 /etc/network/if-up.d/custom-routes.sh
nano /etc/network/if-up.d/custom-routes.sh

Content:

Code:

#!/bin/bash
[ "$IFACE"   = "eth0" ] || exit 0  # we only want interface "eth0"
[ "$ADDRFAM" = "inet" ] || exit 0  # we only want IPv4 (not "inet6" for IPv6)
# add manual routes to old subnets
/usr/sbin/ip route add 192.168.4.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.41.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.42.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.44.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.46.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.47.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.50.0/24 via 192.168.43.1 dev eth0 || true

Code:

touch /etc/network/if-down.d/custom-routes.sh
chown root:root /etc/network/if-down.d/custom-routes.sh
chmod 750 /etc/network/if-down.d/custom-routes.sh
nano /etc/network/if-down.d/custom-routes.sh

Content:

Code:

#!/bin/bash
[ "$IFACE"   = "eth0" ] || exit 0  # we only want interface "eth0"
[ "$ADDRFAM" = "inet" ] || exit 0  # we only want IPv4 (not "inet6" for IPv6)
# remove manual routes to old subnets
/usr/sbin/ip route delete 192.168.4.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route delete 192.168.41.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route delete 192.168.42.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route delete 192.168.44.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route delete 192.168.46.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route delete 192.168.47.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route delete 192.168.50.0/24 via 192.168.43.1 dev eth0 || true

ip a in LXC:

Code:

root@ZabbixLXC:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0@if355: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0e:98:b0:72:0d:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.43.70/24 brd 192.168.43.255 scope global eth0
       valid_lft forever preferred_lft forever
3: eth1@if359: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:c0:91:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.60.175.10/16 brd 10.60.255.255 scope global eth1
       valid_lft forever preferred_lft forever
4: eth2@if363: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:a9:1f:c9 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.61.175.10/16 brd 10.61.255.255 scope global eth2
       valid_lft forever preferred_lft forever

ip route show in LXC:

Code:

root@ZabbixLXC:~# ip route show
default via 10.60.0.1 dev eth1 onlink
10.60.0.0/16 dev eth1 proto kernel scope link src 10.60.175.10
10.61.0.0/16 dev eth2 proto kernel scope link src 10.61.175.10
192.168.43.0/24 dev eth0 proto kernel scope link src 192.168.43.70

Some LXC logs that might be related:

Code:

...
Feb 25 23:20:46 ZabbixLXC systemd[1]: networking.service: Main process exited, code=exited, status=1/FAILURE
Feb 25 23:20:46 ZabbixLXC systemd[1]: networking.service: Failed with result 'exit-code'.
Feb 25 23:20:46 ZabbixLXC systemd[1]: Failed to start networking.service - Raise network interfaces.
Feb 25 23:20:46 ZabbixLXC systemd[1]: Reached target network.target - Network.
...
Feb 25 23:25:47 ZabbixLXC systemd[1]: ifupdown-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Feb 25 23:25:47 ZabbixLXC systemd[1]: ifupdown-wait-online.service: Failed with result 'exit-code'.
Feb 25 23:25:47 ZabbixLXC systemd[1]: Failed to start ifupdown-wait-online.service - Wait for network to be configured by ifupdown.
Feb 25 23:25:47 ZabbixLXC systemd[1]: Reached target network-online.target - Network is Online.
...

Directly pasting...

Code:

/usr/sbin/ip route add 192.168.4.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.41.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.42.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.44.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.46.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.47.0/24 via 192.168.43.1 dev eth0 || true
/usr/sbin/ip route add 192.168.50.0/24 via 192.168.43.1 dev eth0 || true

...into the shell will add the routes:

Code:

root@ZabbixLXC:~# ip route show
default via 10.60.0.1 dev eth1 onlink
10.60.0.0/16 dev eth1 proto kernel scope link src 10.60.175.10
10.61.0.0/16 dev eth2 proto kernel scope link src 10.61.175.10
192.168.4.0/24 via 192.168.43.1 dev eth0
192.168.41.0/24 via 192.168.43.1 dev eth0
192.168.42.0/24 via 192.168.43.1 dev eth0
192.168.43.0/24 dev eth0 proto kernel scope link src 192.168.43.70
192.168.44.0/24 via 192.168.43.1 dev eth0
192.168.46.0/24 via 192.168.43.1 dev eth0
192.168.47.0/24 via 192.168.43.1 dev eth0
192.168.50.0/24 via 192.168.43.1 dev eth0

Any ideas why it isn't working?
Is "eth0@if355" maybe not matching "[ "$IFACE" = "eth0" ]"?
Or is there something working differently because of the containerization?

_gabriel · Feb 26, 2024

I will try replace eth0 with $IFACE
+ echo $IFACE >>/log.ifup

Dunuin · Feb 26, 2024

Nope. Adding a ...

Code:

echo "Script triggered" >> /tmp/ifup.log
echo $IFACE >> /tmp/ifup.log

.. to the top of the script won't create any logfile which could point out what interface name is used. So the script will probably not run in the first place.

DSSTNY · Mar 1, 2024

Dunuin said:
Nope. Adding a ...

Code:

echo "Script triggered" >> /tmp/ifup.log echo $IFACE >> /tmp/ifup.log

.. to the top of the script won't create any logfile which could point out what interface name is used. So the script will probably not run in the first place.

Try without ".sh" at the end of your scripts.
ref : https://manpages.debian.org/bookwor...ory path, shell wildcards may be used as well.

Dunuin · Mar 1, 2024

DSSTNY said:
Try without ".sh" at the end of your scripts.
ref : https://manpages.debian.org/bookworm/ifupdown/interfaces.5.en.html#INCLUDING_OTHER_FILES:~:text=names of the files are checked to match the following regular expression: ^[a-zA-Z0-9_-]+$. In other words, the names must consist entirely of ASCII upper- and lower-case letters, ASCII digits, ASCII underscores, and ASCII minus-hyphens. In the directory path, shell wildcards may be used as well.

could try that but same OS but VM instead of LXC is working perfectly fine ending with ".sh".

ce3rd · Mar 2, 2024

hi, if you found a way to do it from PVE-Host you could use a hookscript -> "It will be called during various phases of the guests lifetime. For an exampleand documentation see the example script under/usr/share/pve-docs/examples/guest-example-hookscript.pl."

Search

Search

Permanent routing in Debian 11 LXC container

m3a2r1

Active Member

Moayad

Proxmox Staff Member

m3a2r1

Active Member

Moayad

Proxmox Staff Member

m3a2r1

Active Member

Dunuin

Distinguished Member

_gabriel

Renowned Member

Dunuin

Distinguished Member

Dunuin

Distinguished Member

_gabriel

Renowned Member

Dunuin

Distinguished Member

DSSTNY

New Member

Dunuin

Distinguished Member

ce3rd

Renowned Member