SNAT stops VM internet access after some time

[yogita]

Hi,

I have a very simple setup as I am still learning about Proxmox and SNAT. We have 1 host in Proxmox Datacenter and the host has 1 VM and 1 container on it. I have setup SNAT as follows -

Zone name : DHCP

vnet name : vnet0

subnet : 10.10.50.0/24 and

IP range: 10.10.50.2 to 10.10.50.5

( I have tried other IP ranges as well and I get the same result)

My VM and container both have internet access for a week and then they suddenly loose internet access, I have to then blow away the subnet and IP range and give it a new one and then it works again for a while before I lose internet access on the VM again ... this has happened 3 times in a row now. I have no firewall rules configured. Need some expert help in helping me configure this correctly so I do not periodically loose internet access on my VMs.

I have followed this support article to setup SNAT - https://pve.proxmox.com/wiki/Setup_Simple_Zone_With_SNAT_and_DHCP

Thanks,

-Yogita.

 

[lethargos]

Is the firewall enabled by any chance, even if you don't have any rules set up? Datacenter/VM level/network interface level?

In my case SNAT doesn't work at all when I enable the firewall (version 8.2.7, kernel version 6.8.12-1)

 

[yogita]

Is the firewall enabled by any chance, even if you don't have any rules set up? Datacenter/VM level/network interface level?

In my case SNAT doesn't work at all when I enable the firewall (version 8.2.7, kernel version 6.8.12-1)

I have no firewall rules configured.

 

[lethargos]

I think you've misread the question. I am asking if the firewall is enabled/on and I'm making a clear distinction between configuring firewall rules and enabling the firewall.

I don't have any firewall rules, but the firewall is enabled.

 

[yogita]

Apologies ... yes it is on ....

 

[lethargos]

Ok, try disabling it (if it's safe to do that) and it might work. At least to test if this is the cause.
There's also the VM-level firewall - you select the VM (Firewall – Options) and the network interface FW – you select the VM (Hardware – Network device, click on it, uncheck "Firewall"), in case you've enabled those too.

What kernel version and proxmox version do you use?
pveversion

 

[yogita]

Ok, try disabling it (if it's safe to do that) and it might work. At least to test if this is the cause.
There's also the VM-level firewall - you select the VM (Firewall – Options) and the network interface FW – you select the VM (Hardware – Network device, click on it, uncheck "Firewall"), in case you've enabled those too.

What kernel version and proxmox version do you use?
pveversion

Ok will try to disable the firewall ... thank you very much for your help ... will let you know if I see any changes after disabling the firewall ...

 

[yogita]

Firewall is now disabled at Datacenter, Node and VM level (earlier it was only disabled at the datacenter and VM level)

 

[yogita]

Disabling firewall at datacenter, node and VM level has not helped, I am still facing the same issue where the VM loses internet access.

 

[shanreich]

Can you post the output of the following commands?

cat /etc/network/interfaces
ip r
iptables -t nat -L
iptables-save

 

[yogita]

root@WG24000106:~# ip r
default via NNN.NNN.NNN.NNN dev vmbr3
10.10.50.0/24 dev vnet1 proto kernel scope link src 10.10.50.1
34.120.255.244 dev vmbr3 scope link
192.168.19.0/24 dev vmbr0 proto kernel scope link src 192.168.19.70
192.168.100.1 dev vmbr3 scope link
NNN.NNN.NN.NN/24 dev vmbr3 proto kernel scope link src NNN.NNN.NNN.NNN
NNN.NNN.NNN.NNN dev vmbr3 scope link

 

[yogita]

root@WG24000106:~# cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp5s0
iface enp5s0 inet manual

auto enp2s0
iface enp2s0 inet manual

iface enp3s0 inet manual

iface enp4s0 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.19.70/24
bridge-ports enp5s0
bridge-stp off
bridge-fd 0
#LAN 2

auto vmbr3
iface vmbr3 inet dhcp
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
#WAN 1

auto vmbr4
iface vmbr4 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
#wOS - WAN 2

auto vmbr5
iface vmbr5 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
#wOS - HA

auto vmbr6
iface vmbr6 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
#wOS - MGMT

source /etc/network/interfaces.d/*

 

[yogita]

root@WG24000106:~# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
ts-postrouting all -- anywhere anywhere
SNAT all -- 10.10.50.0/24 anywhere to:NNN.NNN.NNN.NNN

Chain ts-postrouting (1 references)
target prot opt source destination
MASQUERADE all -- anywhere anywhere mark match 0x40000/0xff0000

 

[yogita]

root@WG24000106:~# iptables-save
# Generated by iptables-save v1.8.9 on Wed Oct 9 13:31:34 2024
*raw
REROUTING ACCEPT [830582:603263067]
:OUTPUT ACCEPT [83230:32554519]
-A PREROUTING -i fwbr+ -j CT --zone 1
COMMIT
# Completed on Wed Oct 9 13:31:34 2024
# Generated by iptables-save v1.8.9 on Wed Oct 9 13:31:34 2024
*nat
REROUTING ACCEPT [12626:1711173]
:INPUT ACCEPT [6677:528537]
:OUTPUT ACCEPT [8566:772031]
OSTROUTING ACCEPT [8566:772031]
:ts-postrouting - [0:0]
-A POSTROUTING -j ts-postrouting
-A POSTROUTING -s 10.10.50.0/24 -o vmbr3 -j SNAT --to-source NNN.NNN.NNN.NNN
-A ts-postrouting -m mark --mark 0x40000/0xff0000 -j MASQUERADE
COMMIT
# Completed on Wed Oct 9 13:31:34 2024
# Generated by iptables-save v1.8.9 on Wed Oct 9 13:31:34 2024
*filter
:INPUT ACCEPT [38317:12428843]
:FORWARD ACCEPT [740101:569575763]
:OUTPUT ACCEPT [83105:32541762]
:ts-forward - [0:0]
:ts-input - [0:0]
-A INPUT -j ts-input
-A FORWARD -j ts-forward
-A ts-forward -i tailscale0 -j MARK --set-xmark 0x40000/0xff0000
-A ts-forward -m mark --mark 0x40000/0xff0000 -j ACCEPT
-A ts-forward -s 100.64.0.0/10 -o tailscale0 -j DROP
-A ts-forward -o tailscale0 -j ACCEPT
-A ts-input -s 100.105.118.25/32 -i lo -j ACCEPT
-A ts-input -s 100.115.92.0/23 ! -i tailscale0 -j RETURN
-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP
-A ts-input -i tailscale0 -j ACCEPT
-A ts-input -p udp -m udp --dport 41641 -j ACCEPT
COMMIT
# Completed on Wed Oct 9 13:31:34 2024

 

[shanreich]

Does your ISP give you a static IP or is it dynamic?

This rule here indicates that traffic gets SNATed to 103.228.x.x:

SNAT all -- 10.10.50.0/24 anywhere to:103.228.x.x

But you do not have that IP configured on any interface. So I suspect, since you have DHCP configured on the WAN interface, that your ISP gives you a different IP every week or so and that breaks your SNAT setup

 

[yogita]

Does your ISP give you a static IP or is it dynamic?

This rule here indicates that traffic gets SNATed to 103.228.243.46:

SNAT all -- 10.10.50.0/24 anywhere to:103.228.243.46

But you do not have that IP configured on any interface. So I suspect, since you have DHCP configured on the WAN interface, that your ISP gives you a different IP every week or so and that breaks your SNAT setup

Thank you for looking into the issue ... ISP gives us a dynamic IP

 

[yogita]

But you do not have that IP configured on any interface. - What can we do to fix this?

 

[shanreich]

I think the best solution would be to disable SNAT on the SDN VNet and then create the rule yourself, with MASQUERADE which translates the IP dynamically - so it should be able to cope with changing IP addresses.

So I think the solution would be:

Turn off SNAT in SDN VNet configuration and apply configuration
Flush NAT rules: iptables -t nat -F
Create a new SNAT rule that uses masquerading:

iptables -t nat -A POSTROUTING -s 10.10.50.0/24 -o vmbr3 -j MASQUERADE

 

[yogita]

I think the best solution would be to disable SNAT on the SDN VNet and then create the rule yourself, with MASQUERADE which translates the IP dynamically - so it should be able to cope with changing IP addresses.

So I think the solution would be:

Turn off SNAT in SDN VNet configuration and apply configuration
Flush NAT rules: iptables -t nat -F
Create a new SNAT rule that uses masquerading:

iptables -t nat -A POSTROUTING -s 10.10.50.0/24 -o vmbr3 -j MASQUERADE

Thank you so very much ... we will give this a try and post our results ... fingers crossed ... thank you again for your support!