Static route in proxmox and systemd-networkd failure in container..

B

bdcatpcsd

New Member
Jan 18, 2019
2
0
1
51
5.3 install with xfs (should it matter)

the building the server is located in is a 10.20.32.0/20

there is a router there for internet, 10.20.32.250

there is a router for lan connectivity, 10.20.32.1

I'm looking for a 'best practice' for how proxmox works..

should I put the static route for the entire network on the proxmox box itself, or only on the containers that will use it.

Currently I have this in proxmox (and proxmox works just fine)

auto vmbr0
iface vmbr0 inet static
address 10.20.32.100
netmask 255.255.240.0
gateway 10.20.32.250
bridge-ports eno1
bridge-stp off
bridge-fd 0
up ip route add 10.20.0.0/16 via 10.20.32.1 dev vmbr0
up ip route add 172.16.0.0/16 via 10.20.32.1 dev vmbr0

then I created an archlinux container, which *was* fine, now .. not so much and I can't figure out why.

the archlinux machine installed just fine and was fine *when rebooted*, the network worked.

but now since the machine was updated it doesn't work without manual intervention..

(archlinux container)

[Match]
Name = eth0

[Network]
Description = Interface eth0 autoconfigured by PVE
Address = 10.20.32.101/20
Gateway = 10.20.32.250
DHCP = none
IPv6AcceptRA = false

that is created and re-created by proxmox upon reboot..

and systemctl start systemd-networkd fails upon reboot..

but running /usr/lib/systemd/systemd-networkd gets the network to work..

The eth0.network file that I would like to use is this: (this forum does terrible with pasted lines.. ugh)


cat /root/eth0.network

[Match]
Name=eth0
[Network]
Description="Something"
Address=10.20.32.101/20
Gateway=10.20.32.250
[Route]
Gateway=10.20.32.1
Destination=10.20.0.0/16
[Route]
Gateway=10.20.32.1
Destination=172.16.0.0/16

tl;dr:
How can I get systemd networking working again in arch, and what is the most efficient way for static routes? per container? or on the proxmox system itself.

thanks in advance.
 
bdcatpcsd said:
then I created an archlinux container, which *was* fine, now .. not so much and I can't figure out why.

the archlinux machine installed just fine and was fine *when rebooted*, the network worked.

but now since the machine was updated it doesn't work without manual intervention..
Click to expand...

My best guess is you're running into this upstream bug:

https://bugs.archlinux.org/task/61313
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1811248
https://github.com/lxc/lxc/issues/2778

systemd version is 240? If yes, you can try to downgrade until it's fixed upstream, any version before 240 seems to work for now.

EDIT: I can confirm this version works for now:

Code: 
core systemd 239.303-1 [installed]
 
Last edited:
Following your links it looks like this might also be an lxc issue..

dmesg | grep audit (on the archlinux container)

[archuser@archclnt101 ~]$ dmesg | grep audit
[ 0.036572] audit: initializing netlink subsys (disabled)
[ 0.036572] audit: type=2000 audit(1547825017.036:1): state=initialized audit_enabled=0 res=1
[ 15.056904] audit: type=1400 audit(1547825031.898:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-start" pid=925 comm="apparmor_parser"
[ 15.118049] audit: type=1400 audit(1547825031.958:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default" pid=924 comm="apparmor_parser"
[ 15.118060] audit: type=1400 audit(1547825031.958:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default-cgns" pid=924 comm="apparmor_parser"
[ 15.118062] audit: type=1400 audit(1547825031.958:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default-with-mounting" pid=924 comm="apparmor_parser"
[ 15.118064] audit: type=1400 audit(1547825031.958:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default-with-nesting" pid=924 comm="apparmor_parser"
[ 17.723480] audit: type=1400 audit(1547825034.562:7): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/usr/bin/lxc-start" pid=1361 comm="apparmor_parser"
[ 17.729217] audit: type=1400 audit(1547825034.570:8): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default" pid=1365 comm="apparmor_parser"
[ 17.729229] audit: type=1400 audit(1547825034.570:9): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-cgns" pid=1365 comm="apparmor_parser"
[ 17.729231] audit: type=1400 audit(1547825034.570:10): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-with-mounting" pid=1365 comm="apparmor_parser"
[ 17.729232] audit: type=1400 audit(1547825034.570:11): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-with-nesting" pid=1365 comm="apparmor_parser"
[ 27.294695] audit: type=1400 audit(1547825044.134:12): apparmor="STATUS" operation="profile_load" profile="/usr/bin/lxc-start" name="lxc-101_</var/lib/lxc>" pid=1930 comm="apparmor_parser"
[ 29.643859] audit: type=1400 audit(1547825046.482:13): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/" pid=2132 comm="(networkd)" flags="rw, rslave"
[ 29.671135] audit: type=1400 audit(1547825046.510:14): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/" pid=2145 comm="(networkd)" flags="rw, rslave"
[ 29.699107] audit: type=1400 audit(1547825046.538:15): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/" pid=2152 comm="(networkd)" flags="rw, rslave"
[ 29.727322] audit: type=1400 audit(1547825046.566:16): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/" pid=2162 comm="(networkd)" flags="rw, rslave"
[ 29.755343] audit: type=1400 audit(1547825046.594:17): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/" pid=2172 comm="(networkd)" flags="rw, rslave"

I cheated..

I run this @reboot via cron:

[root@archclnt101 ~]# cat /root/eth0.linkup

ip link set eth0 up
ip addr add 10.20.32.101/20 dev eth0
ip route add default via 10.20.32.250
ip route add 10.20.0.0/16 via 10.20.32.1 dev eth0
ip route add 172.16.0.0/16 via 10.20.32.1 dev eth0

[root@archclnt101 ~]# pacman -Q | grep systemd
libsystemd 240.34-3
systemd 240.34-3
systemd-sysvcompat 240.34-3

and I've reverted the proxmox machine to this config:

auto vmbr0
iface vmbr0 inet static
address 10.20.32.100
netmask 255.255.240.0
gateway 10.20.32.1
bridge-ports eno1
bridge-stp off
bridge-fd 0


Also is the normal from the container?

[archuser@archclnt101 ~]$ tracepath -n 9.9.9.9
1?: [LOCALHOST] pmtu 1500
1: 10.20.32.250 0.420ms
1: 10.20.32.250 0.698ms
2: x.x.x.x 1.117ms
3: y.y.y.y 7.432ms
4: no reply
5: 140.222.1.59 4.299ms
6: 152.179.72.42 3.694ms asymm 5
7: 129.250.6.69 3.211ms asymm 6
8: no reply
9: 129.250.198.150 5.614ms !H
Resume: pmtu 1500


See the double 10.20.32.250 entry..

From the proxmox host it doesn't do that.. only the container..

Thanks in advance
 
bdcatpcsd said:
I cheated..

I run this @reboot via cron:

[root@archclnt101 ~]# cat /root/eth0.linkup

ip link set eth0 up
ip addr add 10.20.32.101/20 dev eth0
ip route add default via 10.20.32.250
ip route add 10.20.0.0/16 via 10.20.32.1 dev eth0
ip route add 172.16.0.0/16 via 10.20.32.1 dev eth0
Click to expand...

My advice would be to avoid playing around with that, especially with a hacky solution like this one.

bdcatpcsd said:
Also is the normal from the container?

[archuser@archclnt101 ~]$ tracepath -n 9.9.9.9
1?: [LOCALHOST] pmtu 1500
1: 10.20.32.250 0.420ms
1: 10.20.32.250 0.698ms
2: x.x.x.x 1.117ms
3: y.y.y.y 7.432ms
4: no reply
5: 140.222.1.59 4.299ms
6: 152.179.72.42 3.694ms asymm 5
7: 129.250.6.69 3.211ms asymm 6
8: no reply
9: 129.250.198.150 5.614ms !H
Resume: pmtu 1500


See the double 10.20.32.250 entry..

From the proxmox host it doesn't do that.. only the container..
Click to expand...

I'm getting differing outputs, sometimes with double entries, sometimes without. I guess this is not a container thing but a tracepath thing.
 
You must log in or register to reply here.
Top Bottom