Internal NAT issues

Premlez · Jan 24, 2020

Because the server has only one external IP, I'm using internal NAT.
The problem is, that I can't ping (reach) domains on my VMs and CTs, but I can ping (reach) digit IPs.
Here's some screenshots:

Here is my interfaces configuration:

Code:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet static
        address  xxx.xxx.xxx.xxx
        netmask  xxx.xxx.xxx.xxx
        gateway  xxx.xxx.xxx.xxx
#External

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr1
iface vmbr1 inet static
        address  10.10.10.1
        netmask  255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE

Any help would be welcome

Chris · Jan 24, 2020

Premlez said:
Because the server has only one external IP, I'm using internal NAT.
The problem is, that I can't ping (reach) domains on my VMs and CTs, but I can ping (reach) digit IPs.
Here's some screenshots:
View attachment 14371

View attachment 14373

Here is my interfaces configuration:

Code:

source /etc/network/interfaces.d/* auto lo iface lo inet loopback auto eno1 iface eno1 inet static address xxx.xxx.xxx.xxx netmask xxx.xxx.xxx.xxx gateway xxx.xxx.xxx.xxx #External iface eno2 inet manual iface eno3 inet manual iface eno4 inet manual auto vmbr1 iface vmbr1 inet static address 10.10.10.1 netmask 255.255.255.0 bridge-ports none bridge-stp off bridge-fd 0 post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE

View attachment 14374

Any help would be welcome

Hi,
do you have the correct nameserver configured? Check cat /etc/resolv.conf and test with dig google.com

matrix · Jan 24, 2020

Hey

Your internet looks fine, u need to see your DNS.
check of file /etc/resolv.conf

Premlez · Jan 24, 2020

Thanks guys for replys!
I checked /etc/resolv.conf on one of the CTs and it returned all DNS servers I specified on Proxmox. All of them, 8.8.8.8, 8.8.4.4 and 1.1.1.1

matrix · Jan 24, 2020

Do command hostname then host -a YourHostname u will get ur DNS ip
Received 97 bytes from 192.X.Y.Z#53 in 0 ms
Then put the IP in /etc/resolv.conf and try again.

Premlez · Jan 24, 2020

matrix said:
Do command hostname then host -a YourHostname u will get ur DNS ip
Received 97 bytes from 192.X.Y.Z#53 in 0 ms
Then put the IP in /etc/resolv.conf and try again.

It returns an error: ;; connection timed out; no servers could be reached

matrix · Jan 24, 2020

Did you tried to Disable IP forwarding?
echo 1 > /proc/sys/net/ipv4/ip_forward

Premlez · Jan 27, 2020

matrix said:
Did you tried to Disable IP forwarding?
echo 1 > /proc/sys/net/ipv4/ip_forward

Nothing changed, still the same.

Chris · Jan 27, 2020

Premlez said:
Nothing changed, still the same.

The routing should be fine as you are able to ping the servers via their IP. Have you enabled the PVE firewall or use another firewall which blocks outgoing DNS requests? If you have the firewall enabled please try to log the packets. Further you could try using tcpdump on the host to see if your DNS requests are coming through. See here for an example on how to do that https://nil.uniza.sk/using-tcpdump-diagnostics-dns-debian/

Premlez · Jan 27, 2020

Chris said:
The routing should be fine as you are able to ping the servers via their IP. Have you enabled the PVE firewall or use another firewall which blocks outgoing DNS requests? If you have the firewall enabled please try to log the packets. Further you could try using tcpdump on the host to see if your DNS requests are coming through. See here for an example on how to do that https://nil.uniza.sk/using-tcpdump-diagnostics-dns-debian/

UFW is set to allow outgoing port 53 from anywhere.
Here's what I get from running tcpdump -i vmbr1 udp port 53:

Code:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:35:45.817737 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:45.817812 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:50.822998 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:35:50.823086 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:53.826248 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:35:53.826330 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:35:59.832468 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:59.832534 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:04.837663 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:36:04.837723 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:07.840865 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:36:07.840910 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:36:13.847164 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:13.847244 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:18.852421 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:18.852482 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:21.855616 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:21.855658 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)
15:36:27.861777 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:27.861854 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:32.866996 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:32.867075 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:35.870203 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:35.870256 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)

Chris · Jan 27, 2020

Premlez said:

UFW is set to allow outgoing port 53 from anywhere.
Here's what I get from running tcpdump -i vmbr1 udp port 53:

Code:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:35:45.817737 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:45.817812 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:50.822998 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:35:50.823086 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:53.826248 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:35:53.826330 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:35:59.832468 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:59.832534 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:04.837663 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:36:04.837723 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:07.840865 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:36:07.840910 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:36:13.847164 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:13.847244 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:18.852421 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:18.852482 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:21.855616 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:21.855658 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)
15:36:27.861777 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:27.861854 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:32.866996 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:32.867075 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:35.870203 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:35.870256 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)

Is the same true also after routing and masquerading, i.e. tcpdump -i eno1 udp port 53? It seems you get no response, do you block incomming DNS traffic? Also DNS might use tcp, so you will have to allow tcp port 53 in/out as well.

Premlez · Jan 27, 2020

I've got some news.
After disabling UFW, it stared working.
But when I add rule DNS, it still doesn't work. Should I use UFW, or any other Firewall?

Chris · Jan 27, 2020

Which firewall you wish to use is in the end up to you, but you will have to configure it so that you allow the traffic needed for DNS to work.

Premlez · Jan 27, 2020

This is my current UFW status:

Code:

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere                
DNS                        ALLOW       Anywhere                
80/tcp                     ALLOW       Anywhere                
443/tcp                    ALLOW       Anywhere                
22/tcp (v6)                ALLOW       Anywhere (v6)            
DNS (v6)                   ALLOW       Anywhere (v6)            
80/tcp (v6)                ALLOW       Anywhere (v6)            
443/tcp (v6)               ALLOW       Anywhere (v6)

Is there any other rule I need to add?

Chris · Jan 27, 2020

Premlez said:

This is my current UFW status:

Code:

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere               
DNS                        ALLOW       Anywhere               
80/tcp                     ALLOW       Anywhere               
443/tcp                    ALLOW       Anywhere               
22/tcp (v6)                ALLOW       Anywhere (v6)           
DNS (v6)                   ALLOW       Anywhere (v6)           
80/tcp (v6)                ALLOW       Anywhere (v6)           
443/tcp (v6)               ALLOW       Anywhere (v6)

Is there any other rule I need to add?

I am not familiar with ufw syntax, but a quick google search spits out this https://unix.stackexchange.com/questions/131332/ufw-is-blocking-dns

Premlez · Jan 27, 2020

Chris said:
I am not familiar with ufw syntax, but a quick google search spits out this https://unix.stackexchange.com/questions/131332/ufw-is-blocking-dns

Nothing helps. UFW is just blocking everything...
I've tried everything... Allowing all out ports for DNS on vmbr1, on eno1, even tried: iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT, but it doesn't work.

Chris · Jan 28, 2020

Check where the packets get dropped. Do they reach eno1 after routing or are the already dropped at the bridge? Use the logging feature of the firewall if usefull or tools such as tcpdump. Please post the output of iptables-save in order to see the actual rules.

Search

Search

Internal NAT issues

Premlez

New Member

Attachments

Chris

Proxmox Staff Member

matrix

Active Member

Premlez

New Member

matrix

Active Member

Premlez

New Member

matrix

Active Member

Premlez

New Member

Chris

Proxmox Staff Member

Premlez

New Member

Chris

Proxmox Staff Member

Premlez

New Member

Chris

Proxmox Staff Member

Premlez

New Member

Chris

Proxmox Staff Member

Premlez

New Member

Chris

Proxmox Staff Member