[SOLVED] SNAT with IP aliasing not working

murda · Jul 8, 2024

Hi all, I'm trying to achieve the following:

I have 2 public IPv4 addresses
- 1.1.1.1 for the PVE host on enp0s31f6
- 1.1.1.2 for LXC & VMs on enp0s31f6:0
I have a vmbr0 which must translate packets originating from 10.0.0.0/24 to 1.1.1.2 on enp0s31f6. This is so that containers and VMs are reachable to and from the internet via 1.1.1.2.

However, my containers can't reach the internet at all and I'm at my wits end.

Can't reach the internet but I can ping the gateway in the container:

Code:

root@CT100:~# curl ifconfig.me
curl: (28) Failed to connect to ifconfig.me port 80 after 135718 ms: Couldn't connect to server
root@CT100:~# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.040 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.040 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.040 ms
^C
--- 10.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2027ms
rtt min/avg/max/mdev = 0.040/0.040/0.040/0.000 ms

Here is my /etc/network/interfaces:

Code:

auto lo
iface lo inet loopback

auto enp0s31f6
allow-hotplug enp0s31f6
iface enp0s31f6 inet static
    address 1.1.1.1/26
    gateway 1.1.1.3

auto enp0s31f6:0
allow-hotplug enp0s31f6:0
iface enp0s31f6:0 inet static
    address 1.1.1.2/26

auto vmbr0
iface vmbr0 inet static
  address 10.0.0.1/24
  bridge-ports none
  bridge-stp off
  bridge-fd 0

  post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
  post-up iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
  post-down iptables -t nat -D POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2

source /etc/network/interfaces.d/*

shanreich · Jul 8, 2024

If the firewall is enabled on the network interfaces of the Containers (which it is by default), then you need to add the following rules as well

Code:

post-up   iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

murda · Jul 8, 2024

shanreich said:
If the firewall is enabled on the network interfaces of the Containers (which it is by default), then you need to add the following rules as well

Code:

post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

I explicitly disabled the firewall on the container because I had issues with it. I first tried to setup masquerading with 1.1.1.1 which didn't work until I disabled the firewall, despite adding those lines, so for testing I disabled the firewall. But that's a different issue now as I'm trying to get it to work with 1.1.1.2 and the firewall is off (for now)

Edit: for good measure I enabled the firewall on the container, on the node and on datacenter level. And appended those 2 lines to vmbr0.

murda · Jul 8, 2024

ip a

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 90:1b:0e:f1:71:e2 brd ff:ff:ff:ff:ff:ff
    inet 1.1.1.1/26 scope global enp0s31f6
       valid_lft forever preferred_lft forever
    inet 1.1.1.2/26 scope global secondary enp0s31f6
       valid_lft forever preferred_lft forever
    inet6 fe80::921b:eff:fef1:71e2/64 scope link
       valid_lft forever preferred_lft forever
3: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc pfifo_fast state UNKNOWN group default qlen 500
    link/none
    inet 100.112.62.81/32 scope global tailscale0
       valid_lft forever preferred_lft forever
    inet6 fd7a:115c:a1e0::2701:3e51/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::72a6:58a6:44cb:6aef/64 scope link stable-privacy
       valid_lft forever preferred_lft forever
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:b6:33:6d:25:b5 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::289d:c6ff:fe77:fa4a/64 scope link
       valid_lft forever preferred_lft forever
5: veth100i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:b6:33:6d:25:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0

ip r

Code:

ip r
default via 1.1.1.2 dev enp0s31f6 proto kernel onlink
10.0.0.0/24 dev vmbr0 proto kernel scope link src 10.0.0.1
1.1.1.2/26 dev enp0s31f6 proto kernel scope link src 1.1.1.1

iptables-save

Code:

sudo iptables-save
# Generated by iptables-save v1.8.9 on Mon Jul  8 15:47:43 2024
*raw
:PREROUTING ACCEPT [31023:11043045]
:OUTPUT ACCEPT [26197:10750018]
COMMIT
# Completed on Mon Jul  8 15:47:43 2024
# Generated by iptables-save v1.8.9 on Mon Jul  8 15:47:43 2024
*nat
:PREROUTING ACCEPT [1161:65599]
:INPUT ACCEPT [925:54460]
:OUTPUT ACCEPT [628:44662]
:POSTROUTING ACCEPT [863:55741]
:ts-postrouting - [0:0]
-A POSTROUTING -j ts-postrouting
-A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
-A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADE
COMMIT
# Completed on Mon Jul  8 15:47:43 2024
# Generated by iptables-save v1.8.9 on Mon Jul  8 15:47:43 2024
*filter
:INPUT ACCEPT [10123:2498700]
:FORWARD ACCEPT [277:13522]
:OUTPUT ACCEPT [26130:10743321]
:ts-forward - [0:0]
:ts-input - [0:0]
-A INPUT -j ts-input
-A FORWARD -j ts-forward
-A ts-forward -i tailscale0 -j MARK --set-xmark 0x40000/0xff0000
-A ts-forward -m mark --mark 0x40000/0xff0000 -j ACCEPT
-A ts-forward -s 100.64.0.0/10 -o tailscale0 -j DROP
-A ts-forward -o tailscale0 -j ACCEPT
-A ts-input -s 100.112.62.81/32 -i lo -j ACCEPT
-A ts-input -s 100.115.92.0/23 ! -i tailscale0 -j RETURN
-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP
-A ts-input -i tailscale0 -j ACCEPT
-A ts-input -p udp -m udp --dport 41641 -j ACCEPT
COMMIT
# Completed on Mon Jul  8 15:47:43 2024

shanreich · Jul 8, 2024

Are you actually using 1.1.1.1/26 as an IP or is this substituted for your actual IP range by you?

Can you post the output of

Code:

sudo iptables-save -t nat
sudo iptables-save -t raw

murda · Jul 8, 2024

shanreich said:
Are you actually using 1.1.1.1/26 as an IP or is this substituted for your actual IP range by you?

Can you post the output of

Code:

sudo iptables-save -t nat sudo iptables-save -t raw

It's a substitute for this post, to mask my real IP. But both IP addresses are in the same subnet and share default gateway. Should I edit it to e.g. x.x.x.79 and such for clarity?

Code:

# Generated by iptables-save v1.8.9 on Mon Jul  8 16:16:21 2024
*nat
:PREROUTING ACCEPT [1353:79155]
:INPUT ACCEPT [274:17308]
:OUTPUT ACCEPT [602:40852]
:POSTROUTING ACCEPT [752:47683]
:ts-postrouting - [0:0]
-A POSTROUTING -j ts-postrouting
-A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
-A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
-A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADE
COMMIT
# Completed on Mon Jul  8 16:16:21 2024
# Generated by iptables-save v1.8.9 on Mon Jul  8 16:16:21 2024
*raw
:PREROUTING ACCEPT [19320:6931071]
:OUTPUT ACCEPT [16190:10150533]
-A PREROUTING -i fwbr+ -j CT --zone 1
COMMIT
# Completed on Mon Jul  8 16:16:21 2024

murda · Jul 8, 2024

Firewall in PVE webui is enabled on the container, node and datacenter, but no rules are set.

I'm on Hetzner and the stateless firewall is setup to allow all incoming/outgoing:

shanreich · Jul 8, 2024

Seems like there are additional rules (by tailscale?):

Code:

-A POSTROUTING -j ts-postrouting
[...]
-A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADE

So it's jumping to MASQUERADE instead of your SNAT rule and marking the packet. Not sure if tailscale is doing anything else with it. Can you try inserting the rule at the beginning via:

Code:

-I POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2

murda · Jul 8, 2024

shanreich said:
Seems like there are additional rules (by tailscale?):

Code:

-A POSTROUTING -j ts-postrouting [...] -A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADE

So it's jumping to MASQUERADE instead of your SNAT rule and marking the packet. Not sure if tailscale is doing anything else with it. Can you try inserting the rule at the beginning via:

Code:

-I POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2

Not sure if that MASQUERADE rules comes from Tailscale... But I added the rule and still can't reach the internet from my container. I tried it again after disabling all firewalls but still none.

Also might I add I installed Tailscale on the PVE host directly instead of a container with subnet routing.

Code:

sudo iptables -t nat -I POSTROUTING 1 -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2

sudo iptables-save -t nat
sudo iptables-save -t raw
# Generated by iptables-save v1.8.9 on Mon Jul  8 17:10:44 2024
*nat
:PREROUTING ACCEPT [130:7446]
:INPUT ACCEPT [75:4516]
:OUTPUT ACCEPT [62:4274]
:POSTROUTING ACCEPT [80:5087]
:ts-postrouting - [0:0]
-A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
-A POSTROUTING -j ts-postrouting
-A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
-A POSTROUTING -s 10.0.0.0/24 -o enp0s31f6 -j SNAT --to-source 1.1.1.2
-A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADE
COMMIT
# Completed on Mon Jul  8 17:10:44 2024
# Generated by iptables-save v1.8.9 on Mon Jul  8 17:10:44 2024
*raw
:PREROUTING ACCEPT [26470:9385317]
:OUTPUT ACCEPT [23736:14596234]
-A PREROUTING -i fwbr+ -j CT --zone 1
COMMIT
# Completed on Mon Jul  8 17:10:44 2024

curl ifconfig.me
^C
ping ifconfig.me
PING ifconfig.me (34.117.118.44) 56(84) bytes of data.
^C
--- ifconfig.me ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3055ms

shanreich · Jul 8, 2024

murda said:
Not sure if that MASQUERADE rules comes from Tailscale...

I'm pretty sure it is [1]

Can you check via tcpdump whether packets are leaving your interface?

Code:

tcpdump -envi enp0s31f6 icmp

[1] https://github.com/tailscale/tailscale/blob/main/wgengine/router/router_linux_test.go#L120

murda · Jul 8, 2024

shanreich said:
I'm pretty sure it is [1]

Can you check via tcpdump whether packets are leaving your interface?

Code:

tcpdump -envi enp0s31f6 icmp

[1] https://github.com/tailscale/tailscale/blob/main/wgengine/router/router_linux_test.go#L120

Yes, I disabled Tailscale with `tailscale down` and the rule disappeared. Then rebooted just to be sure.

I substituted the addresses with 1.1.1.1 and 1.1.1.2 appropriately, but upon doing so I noticed "ICMP host x.x.x.194 unreachable". I don't recognize this .194 address.

Code:

tcpdump: listening on enp0s31f6, link-type EN10MB (Ethernet), snapshot length 262144 bytes
17:44:02.266065 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 82: (tos 0xc0, ttl 64, id 16987, offset 0, flags [none], proto ICMP (1), length 68)
    1.1.1.1 > 194.26.135.113: ICMP host x.x.x.194 unreachable, length 48
    (tos 0x0, ttl 245, id 6729, offset 0, flags [none], proto TCP (6), length 40)
    194.26.135.113.47888 > x.x.x.194.30624: Flags [S], cksum 0x74e5 (correct), seq 4100925483, win 1024, length 0
17:44:12.890072 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 82: (tos 0xc0, ttl 64, id 23443, offset 0, flags [none], proto ICMP (1), length 68)
    1.1.1.1 > 35.237.11.114: ICMP host x.x.x.194 unreachable, length 48
    (tos 0x0, ttl 250, id 48702, offset 0, flags [none], proto TCP (6), length 40)
    35.237.11.114.45432 > x.x.x.194.8873: Flags [S], cksum 0x49ed (correct), seq 3875133781, win 1024, length 0
17:44:16.218149 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 86: (tos 0xc0, ttl 64, id 56978, offset 0, flags [none], proto ICMP (1), length 72)
    1.1.1.1 > 170.187.163.117: ICMP host x.x.x.194 unreachable, length 52
    (tos 0x0, ttl 241, id 54321, offset 0, flags [none], proto TCP (6), length 44)
    170.187.163.117.49557 > x.x.x.194.8443: Flags [S], cksum 0x46bb (correct), seq 1649156152, win 65535, options [mss 1460], length 0
17:44:25.498173 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 82: (tos 0xc0, ttl 64, id 14229, offset 0, flags [none], proto ICMP (1), length 68)
    1.1.1.1 > 122.194.13.49: ICMP host x.x.x.194 unreachable, length 48
    (tos 0x0, ttl 237, id 54321, offset 0, flags [none], proto TCP (6), length 40)
    122.194.13.49.11874 > x.x.x.194.2222: Flags [S], cksum 0xb494 (correct), seq 1482900930, win 65535, length 0
17:44:28.762008 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 82: (tos 0xc0, ttl 64, id 2497, offset 0, flags [none], proto ICMP (1), length 68)
    1.1.1.1 > 5.188.206.22: ICMP host x.x.x.194 unreachable, length 48
    (tos 0x0, ttl 247, id 41096, offset 0, flags [none], proto TCP (6), length 40)
    5.188.206.22.42556 > x.x.x.194.60140: Flags [S], cksum 0x5a6b (correct), seq 1051725734, win 1024, length 0
17:44:28.762015 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 82: (tos 0xc0, ttl 64, id 49944, offset 0, flags [none], proto ICMP (1), length 68)
    1.1.1.1 > 79.124.62.134: ICMP host x.x.x.194 unreachable, length 48
    (tos 0x0, ttl 247, id 59000, offset 0, flags [none], proto TCP (6), length 40)
    79.124.62.134.59163 > x.x.x.194.9389: Flags [S], cksum 0x599e (correct), seq 2470217495, win 1024, length 0
17:44:37.994729 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61273, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 1, length 64
17:44:39.002211 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61280, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 2, length 64
17:44:39.066162 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 82: (tos 0xc0, ttl 64, id 59462, offset 0, flags [none], proto ICMP (1), length 68)
    1.1.1.1 > 79.124.62.126: ICMP host x.x.x.194 unreachable, length 48
    (tos 0x0, ttl 247, id 9932, offset 0, flags [none], proto TCP (6), length 40)
    79.124.62.126.59160 > x.x.x.194.3889: Flags [S], cksum 0xdea3 (correct), seq 1050045503, win 1024, length 0
17:44:40.026221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61614, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 3, length 64
17:44:41.050226 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 62530, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 4, length 64
17:44:42.074209 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63292, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 5, length 64
17:44:43.098221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63860, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 6, length 64
^C
13 packets captured
13 packets received by filter
0 packets dropped by kernel

shanreich · Jul 8, 2024

Not sure about the other packets, but it seems like the packets are properly leaving your host:

Code:

17:44:40.026221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61614, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 3, length 64
17:44:41.050226 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 62530, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 4, length 64
17:44:42.074209 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63292, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 5, length 64
17:44:43.098221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63860, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 6, length 64

Which would point to an issue outside of PVE (firewall, for instance)

murda · Jul 8, 2024

shanreich said:

Not sure about the other packets, but it seems like the packets are properly leaving your host:

Code:

17:44:40.026221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61614, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 3, length 64
17:44:41.050226 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 62530, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 4, length 64
17:44:42.074209 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63292, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 5, length 64
17:44:43.098221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63860, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 6, length 64

Which would point to an issue outside of PVE (firewall, for instance)

The other packets are when I tried to ping google.com, ifconfig.me and 10.0.0.1.

If I understand correctly the packets do leave, but incoming responses don't arrive?

I think I've configured the firewall in Proxmox correctly. I've only set rules on datacenter level. If it's indeed set correctly, then it does make sense that the issue is external from Proxmox, so at least I know where to look further. I'll probably contact Hetzner regarding this.

Thank you for your help.

murda · Jul 8, 2024

shanreich said:

Not sure about the other packets, but it seems like the packets are properly leaving your host:

Code:

17:44:40.026221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61614, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 3, length 64
17:44:41.050226 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 62530, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 4, length 64
17:44:42.074209 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63292, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 5, length 64
17:44:43.098221 90:1b:0e:f1:71:e2 > 40:71:83:a5:ea:7c, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63860, offset 0, flags [DF], proto ICMP (1), length 84)
    1.1.1.2 > 8.8.8.8: ICMP echo request, id 60534, seq 6, length 64

Which would point to an issue outside of PVE (firewall, for instance)

On second thought, I'm not sure that the packets leave the host. I set up another VPS (another physical machine) and logged both incoming icmp and tcp with tcpdump but packets originating from my Proxmox host aren't reaching the VPS.

shanreich · Jul 9, 2024

murda said:
On second thought, I'm not sure that the packets leave the host. I set up another VPS (another physical machine) and logged both incoming icmp and tcp with tcpdump but packets originating from my Proxmox host aren't reaching the VPS.

They show up in tcpdump on the physical interface, so it's quite certain they are leaving the kernel at least. There's a gateway / firewall /... inbetween where the packet could get lost on the way to your VPS. Do you maybe need to configure a MAC address for the IP address with your hoster?

Can you ping from the host using the alternate IP address?

Code:

ping -S 1.1.1.2 8.8.8.8

murda · Jul 9, 2024

They show up in tcpdump on the physical interface, so it's quite certain they are leaving the kernel at least. There's a gateway / firewall /... inbetween where the packet could get lost on the way to your VPS. Do you maybe need to configure a MAC address for the IP address with your hoster?

Can you ping from the host using the alternate IP address?

Code:

ping -S 1.1.1.2 8.8.8.8

Hi, thanks for your reply.

I think you mean

Code:

ping -I 1.1.1.2 8.8.8.8

? -S is to specify size whereas -I the interface.

Anyhow,

Code:

ping -S 1.1.1.2 8.8.8.8

from the host doesn't seem to work either:

Code:

ping -I 1.1.1.2 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 1.1.1.2 : 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4082ms

And appropriate tcpdump:

Code:

sudo tcpdump -i enp0s31f6 src 1.1.1.2 and icmp -vvv
tcpdump: listening on enp0s31f6, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:49:06.407719 IP (tos 0x0, ttl 64, id 7205, offset 0, flags [DF], proto ICMP (1), length 84)
    static.2.1.1.1.clients.your-server.de > dns.google: ICMP echo request, id 29539, seq 1, length 64
14:49:07.418191 IP (tos 0x0, ttl 64, id 8216, offset 0, flags [DF], proto ICMP (1), length 84)
    static.2.1.1.1.clients.your-server.de > dns.google: ICMP echo request, id 29539, seq 2, length 64
14:49:08.442205 IP (tos 0x0, ttl 64, id 8891, offset 0, flags [DF], proto ICMP (1), length 84)
    static.2.1.1.1.clients.your-server.de > dns.google: ICMP echo request, id 29539, seq 3, length 64
14:49:09.466206 IP (tos 0x0, ttl 64, id 8963, offset 0, flags [DF], proto ICMP (1), length 84)
    static.2.1.1.1.clients.your-server.de > dns.google: ICMP echo request, id 29539, seq 4, length 64
14:49:10.490188 IP (tos 0x0, ttl 64, id 9918, offset 0, flags [DF], proto ICMP (1), length 84)
    static.2.1.1.1.clients.your-server.de > dns.google: ICMP echo request, id 29539, seq 5, length 64

As for the MAC address, I do have a separate MAC address for 1.1.1.2. Do I need to manually configure that in my network config?

murda · Jul 10, 2024

For anyone stumbling on this in the future, I finally just found out that removing the separate MAC from your additional IP in Hetzner Robot allows your additional IP to be connectable to and from the internet. I have no clue why at the moment, because Hetzner has strict rules on MAC addresses being used on their network; you may only use those that are known and assigned to you by them.

In hindsight it was a very simple solution, but it never dawned on me to remove the MAC address. Anyhow, I'm very happy because I can go to sleep now

Code:

ping -I 1.1.1.2 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 1.1.1.2 : 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=5.12 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=5.22 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=5.24 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=5.22 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=119 time=5.21 ms
^C
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 5.123/5.201/5.235/0.040 ms

Search

Search

[SOLVED] SNAT with IP aliasing not working

murda

Member

shanreich

Proxmox Staff Member

murda

Member

murda

Member

shanreich

Proxmox Staff Member

murda

Member

murda

Member

shanreich

Proxmox Staff Member

murda

Member

shanreich

Proxmox Staff Member

murda

Member

shanreich

Proxmox Staff Member

murda

Member

murda

Member

shanreich

Proxmox Staff Member

murda

Member

murda

Member