Bridge+NAT : masquerade works but not as I would like

[Marc Ballat]

Hi,

I have been struggling to get NAT working for a container with only partial success : it works when I run the command from a shell but does not work when I set it up in /etc/network/interfaces. The host runs debian 8 and proxmox 4.4.12 (upgraded from v3, itself upgraded from v2).

So, as long as I don't run

iptables -t nat -A POSTROUTING -s '172.16.0.0/24' -o vmbr0 -j MASQUERADE

from a shell on the host, I can only ping 172.16.0.1 from the container. No DNS, no ping of an external address.

After I run it, everything works as expected in the container.

There is obviously a lot that I don't understand of what's going on under the hood. I've been trying ifdown and ifup but it says that the interface is not configured for both vmbr0 and vmbr1.

Here is the content of /etc/network/run/ifstat :

# ifdown vmbr1
ifdown: interface vmbr1 not configured

Here is my /etc/network/interfaces :

auto lo
iface lo inet loopback

iface eth0 inet manual
post-up /home/ballama/test.sh eth0

auto vmbr0
iface vmbr0 inet static
        address  94.23.35.223
        netmask  255.255.255.0
        gateway  94.23.35.254
        broadcast  94.23.35.255
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        network 94.23.35.0
iface vmbr0 inet6 static
        address  2001:41D0:2:24df::1
        netmask  64
        post-up /sbin/ip -f inet6 route add 2001:41D0:2:24ff:ff:ff:ff:ff dev vmbr0
        post-up /sbin/ip -f inet6 route add default via 2001:41D0:2:24ff:ff:ff:ff:ff
        pre-down /sbin/ip -f inet6 route del default via 2001:41D0:2:24ff:ff:ff:ff:ff
        pre-down /sbin/ip -f inet6 route del 2001:41D0:2:24ff:ff:ff:ff:ff dev vmbr0

auto vmbr1
iface vmbr1 inet static
        address  172.16.0.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        post-up /etc/pve/kvm-networking.sh
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up /sbin/iptables -t nat -A POSTROUTING -s '172.16.0.0/24' -o vmbr0 -j MASQUERADE
        post-down /sbin/iptables -t nat -D POSTROUTING -s '172.16.0.0/24' -o vmbr0 -j MASQUERADE

And here is the output of ifconfig :

eth0      Link encap:Ethernet  HWaddr 4c:72:b9:43:e9:1c 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5982 errors:0 dropped:33 overruns:0 frame:0
          TX packets:4300 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1866234 (1.7 MiB)  TX bytes:896119 (875.1 KiB)
          Interrupt:20 Memory:fe500000-fe520000

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:308 errors:0 dropped:0 overruns:0 frame:0
          TX packets:308 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:35254 (34.4 KiB)  TX bytes:35254 (34.4 KiB)

tap100i0  Link encap:Ethernet  HWaddr 4a:c3:14:a3:e5:b0 
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:1040 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3722 errors:0 dropped:95 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:89951 (87.8 KiB)  TX bytes:1272888 (1.2 MiB)

tap103i0  Link encap:Ethernet  HWaddr 8e:27:49:c9:f3:ac 
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:829 errors:0 dropped:0 overruns:0 frame:0
          TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:73515 (71.7 KiB)  TX bytes:8827 (8.6 KiB)

vmbr0     Link encap:Ethernet  HWaddr 4c:72:b9:43:e9:1c 
          inet addr:94.23.35.223  Bcast:94.23.35.255  Mask:255.255.255.0
          inet6 addr: fe80::4e72:b9ff:fe43:e91c/64 Scope:Link
          inet6 addr: 2001:41d0:2:24df::1/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2602 errors:0 dropped:3 overruns:0 frame:0
          TX packets:3238 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:619249 (604.7 KiB)  TX bytes:782906 (764.5 KiB)

vmbr1     Link encap:Ethernet  HWaddr 8e:27:49:c9:f3:ac 
          inet addr:172.16.0.1  Bcast:172.16.0.255  Mask:255.255.255.0
          inet6 addr: fe80::5cb4:50ff:fe57:3cf7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:829 errors:0 dropped:0 overruns:0 frame:0
          TX packets:95 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:61909 (60.4 KiB)  TX bytes:13177 (12.8 KiB)

I tried on my own but now I need to call for help ;-)

Marc

 

[manu]

Try the following command, it should display the configuration of your network devices as parsed by the network subsystem.

ifquery vmbr1

your problem the masquerading only works when you manually enter the iptables command after the interfaces have been brought up, would point for a problem in the post-up synthax, but things look also ok here.

what are the extra shells scripts you defined in post-up ? could they cause a problem here ?

 

[Marc Ballat]

Excellent.

I commented out

#post-up /home/ballama/test.sh vmbr1

#post-up /etc/pve/kvm-networking.sh

and it worked. The first script was added to debug : it writes the date and the argument (the name of the interface) to a text file. The second one was added automatically by the GUI when creating the interface. I am not sure which one causes the problem but I thought that all lines in /etc/network/interfaces would be executed, independently of their exit status.

Any way, thanks for the hint ! Would you like me to try commenting out the lines one by one to find out which one is the culprit ?

Marc

 

[manu]

Yes pleas., I am interested to know which script is causing the problem.

Also can you post the content of /etc/pve/kvm-networking.sh ?

This file does not come from Proxmox, I suppose it comes from your hoster ( OVH)

 

[Marc Ballat]

The culprit seems to be

post-up /etc/pve/kvm-networking.sh

.

I do not know where it comes from (I may have copied and pasted it from the OVH tutorial indeed).

Thanks for your help and time.

Marc

 

[manu]

could you post the content of this file, and from which OVH tutorial did you see it mentionned