Internal NAT issues

Premlez

New Member
Jan 24, 2020
8
1
3
20
Because the server has only one external IP, I'm using internal NAT.
The problem is, that I can't ping (reach) domains on my VMs and CTs, but I can ping (reach) digit IPs.
Here's some screenshots:
1579850980486.png

1579850998317.png

Here is my interfaces configuration:
Code:
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet static
        address  xxx.xxx.xxx.xxx
        netmask  xxx.xxx.xxx.xxx
        gateway  xxx.xxx.xxx.xxx
#External

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr1
iface vmbr1 inet static
        address  10.10.10.1
        netmask  255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE
1579851192256.png

Any help would be welcome :)
 

Attachments

Chris

Proxmox Staff Member
Staff member
Jan 2, 2019
598
55
28
Because the server has only one external IP, I'm using internal NAT.
The problem is, that I can't ping (reach) domains on my VMs and CTs, but I can ping (reach) digit IPs.
Here's some screenshots:
View attachment 14371

View attachment 14373

Here is my interfaces configuration:
Code:
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet static
        address  xxx.xxx.xxx.xxx
        netmask  xxx.xxx.xxx.xxx
        gateway  xxx.xxx.xxx.xxx
#External

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr1
iface vmbr1 inet static
        address  10.10.10.1
        netmask  255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE
View attachment 14374

Any help would be welcome :)
Hi,
do you have the correct nameserver configured? Check cat /etc/resolv.conf and test with dig google.com
 
  • Like
Reactions: Premlez

Premlez

New Member
Jan 24, 2020
8
1
3
20
Thanks guys for replys!
I checked /etc/resolv.conf on one of the CTs and it returned all DNS servers I specified on Proxmox. All of them, 8.8.8.8, 8.8.4.4 and 1.1.1.1
 

matrix

Member
Jan 24, 2020
89
9
8
26
Do command hostname then host -a YourHostname u will get ur DNS ip
Received 97 bytes from 192.X.Y.Z#53 in 0 ms
Then put the IP in /etc/resolv.conf and try again.
 
  • Like
Reactions: Premlez

Premlez

New Member
Jan 24, 2020
8
1
3
20
Do command hostname then host -a YourHostname u will get ur DNS ip
Received 97 bytes from 192.X.Y.Z#53 in 0 ms
Then put the IP in /etc/resolv.conf and try again.
It returns an error: ;; connection timed out; no servers could be reached
 

Chris

Proxmox Staff Member
Staff member
Jan 2, 2019
598
55
28
Nothing changed, still the same.
The routing should be fine as you are able to ping the servers via their IP. Have you enabled the PVE firewall or use another firewall which blocks outgoing DNS requests? If you have the firewall enabled please try to log the packets. Further you could try using tcpdump on the host to see if your DNS requests are coming through. See here for an example on how to do that https://nil.uniza.sk/using-tcpdump-diagnostics-dns-debian/
 
  • Like
Reactions: Premlez

Premlez

New Member
Jan 24, 2020
8
1
3
20
The routing should be fine as you are able to ping the servers via their IP. Have you enabled the PVE firewall or use another firewall which blocks outgoing DNS requests? If you have the firewall enabled please try to log the packets. Further you could try using tcpdump on the host to see if your DNS requests are coming through. See here for an example on how to do that https://nil.uniza.sk/using-tcpdump-diagnostics-dns-debian/
UFW is set to allow outgoing port 53 from anywhere.
Here's what I get from running tcpdump -i vmbr1 udp port 53:
Code:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:35:45.817737 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:45.817812 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:50.822998 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:35:50.823086 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:53.826248 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:35:53.826330 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:35:59.832468 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:59.832534 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:04.837663 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:36:04.837723 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:07.840865 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:36:07.840910 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:36:13.847164 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:13.847244 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:18.852421 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:18.852482 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:21.855616 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:21.855658 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)
15:36:27.861777 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:27.861854 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:32.866996 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:32.867075 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:35.870203 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:35.870256 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)
 

Chris

Proxmox Staff Member
Staff member
Jan 2, 2019
598
55
28
UFW is set to allow outgoing port 53 from anywhere.
Here's what I get from running tcpdump -i vmbr1 udp port 53:
Code:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vmbr1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:35:45.817737 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:45.817812 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:50.822998 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:35:50.823086 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:35:53.826248 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:35:53.826330 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:35:59.832468 IP 10.10.10.100.36220 > dns.google.domain: 9545+ A? google.com. (28)
15:35:59.832534 IP 10.10.10.100.36220 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:04.837663 IP 10.10.10.100.46883 > dns.google.domain: 9545+ A? google.com. (28)
15:36:04.837723 IP 10.10.10.100.46883 > dns.google.domain: 62048+ AAAA? google.com. (28)
15:36:07.840865 IP 10.10.10.100.43727 > one.one.one.one.domain: 9545+ A? google.com. (28)
15:36:07.840910 IP 10.10.10.100.43727 > one.one.one.one.domain: 62048+ AAAA? google.com. (28)
15:36:13.847164 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:13.847244 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:18.852421 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:18.852482 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:21.855616 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:21.855658 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)
15:36:27.861777 IP 10.10.10.100.39350 > dns.google.domain: 11839+ A? google.com. (28)
15:36:27.861854 IP 10.10.10.100.39350 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:32.866996 IP 10.10.10.100.53151 > dns.google.domain: 11839+ A? google.com. (28)
15:36:32.867075 IP 10.10.10.100.53151 > dns.google.domain: 56407+ AAAA? google.com. (28)
15:36:35.870203 IP 10.10.10.100.60405 > one.one.one.one.domain: 11839+ A? google.com. (28)
15:36:35.870256 IP 10.10.10.100.60405 > one.one.one.one.domain: 56407+ AAAA? google.com. (28)
Is the same true also after routing and masquerading, i.e. tcpdump -i eno1 udp port 53? It seems you get no response, do you block incomming DNS traffic? Also DNS might use tcp, so you will have to allow tcp port 53 in/out as well.
 
  • Like
Reactions: Premlez

Premlez

New Member
Jan 24, 2020
8
1
3
20
I've got some news.
After disabling UFW, it stared working.
But when I add rule DNS, it still doesn't work. Should I use UFW, or any other Firewall?
 
  • Like
Reactions: Chris

Chris

Proxmox Staff Member
Staff member
Jan 2, 2019
598
55
28
Which firewall you wish to use is in the end up to you, but you will have to configure it so that you allow the traffic needed for DNS to work.
 

Premlez

New Member
Jan 24, 2020
8
1
3
20
This is my current UFW status:
Code:
To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere                
DNS                        ALLOW       Anywhere                
80/tcp                     ALLOW       Anywhere                
443/tcp                    ALLOW       Anywhere                
22/tcp (v6)                ALLOW       Anywhere (v6)            
DNS (v6)                   ALLOW       Anywhere (v6)            
80/tcp (v6)                ALLOW       Anywhere (v6)            
443/tcp (v6)               ALLOW       Anywhere (v6)
Is there any other rule I need to add?
 

Chris

Proxmox Staff Member
Staff member
Jan 2, 2019
598
55
28
This is my current UFW status:
Code:
To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere               
DNS                        ALLOW       Anywhere               
80/tcp                     ALLOW       Anywhere               
443/tcp                    ALLOW       Anywhere               
22/tcp (v6)                ALLOW       Anywhere (v6)           
DNS (v6)                   ALLOW       Anywhere (v6)           
80/tcp (v6)                ALLOW       Anywhere (v6)           
443/tcp (v6)               ALLOW       Anywhere (v6)
Is there any other rule I need to add?
I am not familiar with ufw syntax, but a quick google search spits out this https://unix.stackexchange.com/questions/131332/ufw-is-blocking-dns
 

Chris

Proxmox Staff Member
Staff member
Jan 2, 2019
598
55
28
Check where the packets get dropped. Do they reach eno1 after routing or are the already dropped at the bridge? Use the logging feature of the firewall if usefull or tools such as tcpdump. Please post the output of iptables-save in order to see the actual rules.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!