Masquerading (NAT) with iptables

[svennd]

So I would like to : setup the proxmox with 1 ip, and create lxc containers that have access to internet (network). some of those containers should be accessible from "outside" the box. I used the network model to setup masquerading using iptables. However the containers don't have access to the network nor to the host as far as I can test.
I'm using the latest build (proxmox 4.3, update 27 sept. 2016)

this is /etc/network/interfaces :

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
        netmask  255.255.255.0
        gateway  143.169.254.254

auto vmbr1
iface vmbr1 inet static
        address 10.10.10.254
        netmask 255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        #post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up         iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE
        post-up         iptables -t nat -A POSTROUTING -s '10.10.10.0/24' ! -d '10.10.10.0/24' -o eth0 -j MASQUERADE
        post-down       iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE

Eth0 gets an ip and the host can connect to the network/outside. So I get :

eth0      Link encap:Ethernet  HWaddr 0c:c4:7a:d9:1f:e8
          inet addr:143.169.xx.xx  Bcast:143.169.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2110313 errors:0 dropped:2340 overruns:0 frame:0
          TX packets:14848 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:618930237 (590.2 MiB)  TX bytes:6935397 (6.6 MiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:3437 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3437 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:962705 (940.1 KiB)  TX bytes:962705 (940.1 KiB)

veth100i0 Link encap:Ethernet  HWaddr 1a:4a:9d:25:bb:3f
          inet6 addr: fe80::184a:9dff:fe25:bb3f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:53 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2898 (2.8 KiB)  TX bytes:648 (648.0 B)

vmbr0     Link encap:Ethernet  HWaddr 0c:c4:7a:d9:1f:e8
          inet addr:143.169.xx.xx  Bcast:143.169.255.255  Mask:255.255.0.0
          inet6 addr: fec0::c:ec4:7aff:fed9:1fe8/64 Scope:Site
          inet6 addr: fe80::ec4:7aff:fed9:1fe8/64 Scope:Link
          inet6 addr: 2002:8fa9:34c0:c:ec4:7aff:fed9:1fe8/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:711401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12722 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:51400991 (49.0 MiB)  TX bytes:6819441 (6.5 MiB)

vmbr1     Link encap:Ethernet  HWaddr 0a:8e:bc:3d:30:5d
          inet addr:10.10.10.254  Bcast:10.10.10.255  Mask:255.255.255.0
          inet6 addr: fe80::88e:bcff:fe3d:305d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:578 (578.0 B)

I created a Centos 6 container and gave him to vmbr1, however there seems to be a vmbr0 and veth100i0 coming out of nowhere ?

The container settings :

cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=none
IPADDR=10.10.10.5
NETMASK=255.255.255.0
GATEWAY=10.10.10.254

What did I do wrong, or should I try to get the container network access ?

thanks!

 

[svennd]

Now it seems to work with this setup :

auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet dhcp
        netmask 255.255.255.0
        gateway 143.169.254.254
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.10.10.254
        netmask 255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        #post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up         iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
        post-up         iptables -t nat -A POSTROUTING -s '10.10.10.0/24' ! -d '10.10.10.0/24' -o vmbr0 -j MASQUERADE
        post-down       iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE

 

[svennd]

Seems the fix was temporary, I added

post-up echo 1 > /proc/sys/net/ipv4/ip_forward

but that did not work. The fix stopped working once I added a new container to the machine. Where should I start debugging this ?

 

[svennd]

The problem is the moment I add the second container, the container can't ping the host nor his neighbor, nor the internet....

Anyone ?

 

[svennd]

seems I was setting up vmbr0 to the second container, instead of vmbr1. now both work.