[SOLVED] Two bridges & two different subnets for LXC

[jonsch]

Cheers,

on my Proxmox Host there are two bridges configured, one for public IP's (vmbr0) and one for internal private IP's which are handled with VLAN's (vmbr1).
I have some LXC's which are connected to the public bridge and I want to connect them to the private once too for management stuff.

I've tried so much to get it up and running, it worked in one scenario but the networking service failed everytime.
Is this scenario possible to configure for LXC's ot only for VM's?

When I configure both like the screenshot, the network services fails. I've even tried many other configurations and I can't find it out how to handle it. Don't wonder, just blacked the public IP's adresses...

regards,
Jonas

 

[aaron]

Have you tried leaving the gateway empty for the second NIC? There can only be one default gateway.

 

[jonsch]

So it's right, that in the /etc/network/interfaces is no configuration for the VLAN?

Ok, I've tried this but for my scenario I have to push then a static route, because the 10.10.5.0/24 has to communicate through an IPsec to 192.168.100.0/24, so when I try to push the route for the second interface, the networking service crashing again.

uto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 81.7.x.x/24
        gateway 81.7.x.x

auto eth1
iface eth1 inet static
        address 10.10.5.5/24
        up route add -net 192.168.100.0 netmask 255.255.255.0 gw 10.10.5.1
        down route del -net 192.168.100.0 netmask 255.255.255.0 gw 10.10.5.1

Job for networking.service failed because the control process exited with error code.
See "systemctl status networking.service" and "journalctl -xe" for details.
root@test:~# systemctl status networking
* networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2020-10-30 09:28:11 UTC; 2s ago
     Docs: man:interfaces(5)
  Process: 396 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=1/FAILURE)
Main PID: 396 (code=exited, status=1/FAILURE)

Oct 30 09:28:11 test systemd[1]: Starting Raise network interfaces...
Oct 30 09:28:11 test ifup[396]: RTNETLINK answers: File exists
Oct 30 09:28:11 test ifup[396]: ifup: failed to bring up eth1
Oct 30 09:28:11 test systemd[1]: networking.service: Main process exited, code=exited, status=1/FAILURE
Oct 30 09:28:11 test systemd[1]: networking.service: Failed with result 'exit-code'.
Oct 30 09:28:11 test systemd[1]: Failed to start Raise network interfaces.

 

[jonsch]

Hello,

no one can help me here out?
Maybe @aaron ?

 

[aaron]

Does it work if you remove the up and down commands and add the route manually? If so, try to use post-up and pre-down instead of up and down.

 

[jonsch]

This is my fallback solution at the moment to add the routes manually, yes.

If so, try to use post-up and pre-down instead of up and down.

Same error below.

/etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 81.7.x.x/24
        gateway 81.7.x.x

auto eth1
iface eth1 inet static
        address 10.10.5.5/24
        post-up route add -net 192.168.100.0 netmask 255.255.255.0 gw 10.10.5.1
        pre-down route del -net 192.168.100.0 netmask 255.255.255.0 gw 10.10.5.1

Error

* networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2020-11-02 09:55:13 UTC; 5min ago
     Docs: man:interfaces(5)
  Process: 60 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=1/FAILURE)
 Main PID: 60 (code=exited, status=1/FAILURE)

Nov 02 09:55:12 test systemd[1]: Starting Raise network interfaces...
Nov 02 09:55:13 test ifup[60]: /bin/sh: 1: route: not found
Nov 02 09:55:13 test ifup[60]: ifup: failed to bring up eth1
Nov 02 09:55:13 test systemd[1]: networking.service: Main process exited, code=exited, status=1/FAILURE
Nov 02 09:55:13 test systemd[1]: networking.service: Failed with result 'exit-code'.
Nov 02 09:55:13 test systemd[1]: Failed to start Raise network interfaces.

 

[aaron]

There is now an interesting line in the logs that I see:
> Nov 02 09:55:13 test ifup[60]: /bin/sh: 1: route: not found

Try to modify the call to route to the full path. Should be /usr/sbin/route instead of just route.

 

[jonsch]

Try to modify the call to route to the full path. Should be /usr/sbin/route instead of just route.

The path /usr/sbin/route is not existing, I have found under the path /usr/bin/routef and /usr/bin/routel so I have tried to use the routef because this is the script to flush the routing table.

After edit the /etc/network/interfaces the network service will start correctly but there is no route set and I can't reach the destination subnet.

* networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Mon 2020-11-02 10:23:04 UTC; 6s ago
     Docs: man:interfaces(5)
  Process: 60 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
Main PID: 60 (code=exited, status=0/SUCCESS)

Nov 02 10:23:03 test systemd[1]: Starting Raise network interfaces...
Nov 02 10:23:04 test ifup[60]: Usage: routef
Nov 02 10:23:04 test ifup[60]: This script will flush the IPv4 routing table
Nov 02 10:23:04 test systemd[1]: Started Raise network interfaces.

ip r

default via 81.7.x.x dev eth0 onlink
10.10.5.0/24 dev eth1 proto kernel scope link src 10.10.5.5
81.7.x.x/24 dev eth0 proto kernel scope link src 81.7.x.x

 

[aaron]

routel does not configure new routes but only lists them in a nice readable way.

To get the path to the route cmd run the following on the node: which route.

 

[jonsch]

The output is empty. Maybe there is something missing I have to create for the routing tables to be loaded or something else?

 

[aaron]

Well, it seems that the container does not come with the route command. Then, how do you apply the route manually?
Besides installing it, the container should have the ip command available. You can also add the route with ip route add.
I don't know the full syntax right away but a bit of searching / man page reading should give you the necessary information.

 

[jonsch]

I do not know why?
But I can still use the ip command and I can add static routes with ip route add, so there must be another problem I think so.

 

[aaron]

So you are running ip route .... when adding the route manually?

Then add that command to the up / post-up section of the config, ideally with the full path to ip. Currently, you try to call route (which is not installed) and not ip route there.

 

[jonsch]

I've found it out now, it seems we are both not up to date regarding static routes handling *g*

route is the old traditional tool and available on numerous Unix systems. ip belongs to the iproute2 suite which is a Linux only tool and uses the Netlink API, which is a socket like interface for accessing kernel information about interfaces, address assignments and routes. It replaces most of the functionality of ifconfig, route, netstat and a few others.
I assume you're on Linux, then you should use ip since route and ifconfig are deprecated, although still widely used.
Further reading:

• ifconfig vs iproute2
• iproute2: Life after ifconfig
• Deprecated Linux networking commands and their replacements

I've now changed the configuration as follow. Tthe routes get automatically pushed and the network services seems happy.

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 81.7.x.x/24
        gateway 81.7.x.x

auto eth1
iface eth1 inet static
        address 10.10.5.5/24
        post-up /sbin/ip route add 192.168.100.0/24 via 10.10.5.1
        pre-down /sbin/ip route add 192.168.100.0/24 via 10.10.5.1

Tahnks for your help!

 

[aaron]

Great. Could you please mark the thread as solved? To do so, edit the first post and select the prefix from the dropdown next to the title. Thanks