Proxmox VE 7.2-3 - Host can't reach internet, but VMs can...

bmcgonag

Member
Jul 22, 2021
17
0
6
51
I'm not sure when this happened, but I've noticed that in attempting to update my host, it couldn't reach the repos. I then found that it simply can't reach the internet at all, but all of my VMs / CTs can.

I can reach the host internally on my network via SSH and the Web UI.

I have checked the following:

1. My network is on the 192.168.10.x/24 subnet, and my gateway / router / firewall is 192.168.10.1
Based on this information, here is my /etc/network/interfaces file

Code:
auto lo
iface lo inet loopback

iface eno4 inet manual

iface eno3 inet manual

iface eno1 inet manual

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.10.253/24
        gateway 192.168.10.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

I have the following in my /etc/resolv.conf:
Code:
earch mcghome.localdomain
nameserver 192.168.10.1
nameserver 1.1.1.1
nameserver 1.0.0.1

And my /etc/hosts has the following:
Code:
127.0.0.1 localhost.localdomain localhost
192.168.10.253 aria.mcghome.localdomain aria

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

And my /etc/hostname has the following:
Code:
aria

To me, everything looks correct, and I don't see any reason why it wouldn't be able to reach the internet, but if you see anything, or have ideas on what else I might try, please let me know.
 
hi,

* can you ping external IP addresses such as 1.1.1.1 from your PVE host?

* your /etc/resolv.conf starts with earch (should be "search"). is this a copy/paste mistake or is it really as such inside the file?

* does it make any difference to change the order of the nameservers inside /etc/resolv.conf ?
 
* your /etc/resolv.conf starts with earch (should be "search"). is this a copy/paste mistake or is it really as such inside the file?
Yes, just a copy/paste mistake, it is "search"

* does it make any difference to change the order of the nameservers inside /etc/resolv.conf ?
New order:
Code:
search mcghome.localdomain
nameserver 1.1.1.1
nameserver 1.0.0.1
nameserver 192.168.10.1

Ping Results after change:
Code:
root@aria:~# ping 192.168.10.34 # separate machine
PING 192.168.10.34 (192.168.10.34) 56(84) bytes of data.
64 bytes from 192.168.10.34: icmp_seq=1 ttl=64 time=0.345 ms
64 bytes from 192.168.10.34: icmp_seq=2 ttl=64 time=0.347 ms
64 bytes from 192.168.10.34: icmp_seq=3 ttl=64 time=0.215 ms
64 bytes from 192.168.10.34: icmp_seq=4 ttl=64 time=0.311 ms
64 bytes from 192.168.10.34: icmp_seq=5 ttl=64 time=0.232 ms
^C
--- 192.168.10.34 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4102ms
rtt min/avg/max/mdev = 0.215/0.290/0.347/0.056 ms
root@aria:~#
root@aria:~#
root@aria:~# ping 192.168.10.141 # VM on Host
PING 192.168.10.141 (192.168.10.141) 56(84) bytes of data.
64 bytes from 192.168.10.141: icmp_seq=6 ttl=64 time=0.082 ms
64 bytes from 192.168.10.141: icmp_seq=7 ttl=64 time=0.143 ms
64 bytes from 192.168.10.141: icmp_seq=8 ttl=64 time=0.113 ms
64 bytes from 192.168.10.141: icmp_seq=9 ttl=64 time=0.126 ms
64 bytes from 192.168.10.141: icmp_seq=10 ttl=64 time=0.129 ms
64 bytes from 192.168.10.141: icmp_seq=11 ttl=64 time=0.122 ms
64 bytes from 192.168.10.141: icmp_seq=12 ttl=64 time=0.188 ms
64 bytes from 192.168.10.141: icmp_seq=13 ttl=64 time=0.108 ms
64 bytes from 192.168.10.141: icmp_seq=14 ttl=64 time=0.210 ms
64 bytes from 192.168.10.141: icmp_seq=15 ttl=64 time=0.090 ms
^C
--- 192.168.10.141 ping statistics ---
15 packets transmitted, 10 received, 33.3333% packet loss, time 14325ms
rtt min/avg/max/mdev = 0.082/0.131/0.210/0.038 ms
root@aria:~# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
^C
--- 1.1.1.1 ping statistics ---
61 packets transmitted, 0 received, 100% packet loss, time 61430ms

Ping Results from a different machine on my network:
Code:
brian@brian-dell-main:~$ ping 192.168.10.253 # ping to the proxmox host
PING 192.168.10.253 (192.168.10.253) 56(84) bytes of data.
64 bytes from 192.168.10.253: icmp_seq=1 ttl=64 time=0.220 ms
64 bytes from 192.168.10.253: icmp_seq=2 ttl=64 time=0.338 ms
64 bytes from 192.168.10.253: icmp_seq=3 ttl=64 time=0.201 ms
64 bytes from 192.168.10.253: icmp_seq=4 ttl=64 time=0.287 ms
64 bytes from 192.168.10.253: icmp_seq=5 ttl=64 time=0.193 ms
^C
--- 192.168.10.253 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4095ms
rtt min/avg/max/mdev = 0.193/0.247/0.338/0.055 ms
brian@brian-dell-main:~$ ping 192.168.10.141 # ping to VM on proxmox
PING 192.168.10.141 (192.168.10.141) 56(84) bytes of data.
64 bytes from 192.168.10.141: icmp_seq=1 ttl=64 time=0.377 ms
64 bytes from 192.168.10.141: icmp_seq=2 ttl=64 time=0.430 ms
64 bytes from 192.168.10.141: icmp_seq=3 ttl=64 time=0.425 ms
64 bytes from 192.168.10.141: icmp_seq=4 ttl=64 time=0.263 ms
^C
--- 192.168.10.141 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3073ms
rtt min/avg/max/mdev = 0.263/0.373/0.430/0.067 ms
brian@brian-dell-main:~$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=35.7 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=29.1 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=55 time=32.7 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=55 time=35.9 ms
^C
--- 1.1.1.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 29.051/33.312/35.855/2.768 ms

So, no, no difference.
 
root@aria:~# ping 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. ^C --- 1.1.1.1 ping statistics --- 61 packets transmitted, 0 received, 100% packet loss, time 61430ms
if you can't ping external IP addresses then i would start looking for any routing or firewall related settings that are misconfigured.

* can you ping your gateway 192.168.10.1 from PVE?

* is there a firewall on your gateway? would make sense to check the rules there as well

* is the PVE firewall activated on node or datacenter levels? does stopping it with pve-firewall stop make any difference?
 
* can you ping your gateway 192.168.10.1 from PVE?
No, I can't - odd.

* is there a firewall on your gateway? would make sense to check the rules there as well
Yes, and have now double checked all rules, None for outgoing, only for incoming, and only to port forward to my NGinX Proxy on a completely different machine.

* is the PVE firewall activated on node or datacenter levels? does stopping it with pve-firewall stop make any difference?
The Node Firewall showed 'Yes', but no rules were setup, so I turned it to 'No' in the web ui, and also ran the `pve-firewall stop` command, but still cannot ping my main router, nor any external sites.
 
No, I can't - odd.
once you fix that it'll probably work :)

* how are the guests network configured? are they using the same gateway?
* can you ping the router from inside the guests?

comparing the output of ip route from the PVE host and inside a guest might also shed some light on the situation.

ip addr on the PVE host is probably also worth taking a look
 
* how are the guests network configured? are they using the same gateway?
Yes, they all use the same gateway.

* can you ping the router from inside the guests?
No, also can't ping the router from inside the guests, but I can ping external sites like google.com and 1.1.1.1 from inside the guests.

I have pulled the `ip route` information from this host (aria) and my other proxmox machine host (bia) to compare. I've also compared all network, firewall, and DNS settings between the two to make sure that (other than IP address) they are the same.

ip route:
Area (host having issues)

Code:
default via 192.168.10.1 dev vmbr0 proto kernel onlink
192.168.10.0/24 dev vmbr0 proto kernel scope link src 192.168.10.253

Bia (host not having any issues)

Code:
default via 192.168.10.1 dev vmbr0 proto kernel onlink
192.168.10.0/24 dev vmbr0 proto kernel scope link src 192.168.10.209

ip addr show
Aria (host having issues)
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 24:6e:94:1a:0b:34 brd ff:ff:ff:ff:ff:ff
    altname enp6s0f0
3: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 24:6e:94:1a:0b:35 brd ff:ff:ff:ff:ff:ff
    altname enp6s0f1
4: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether 24:6e:94:1a:0b:30 brd ff:ff:ff:ff:ff:ff
    altname enp1s0f0
5: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 24:6e:94:1a:0b:32 brd ff:ff:ff:ff:ff:ff
    altname enp1s0f1
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 24:6e:94:1a:0b:30 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.253/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::266e:96ff:fe1a:e30/64 scope link
       valid_lft forever preferred_lft forever
7: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000
    link/ether fe:4a:80:47:67:a4 brd ff:ff:ff:ff:ff:ff
8: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether f2:d3:3d:12:2a:e5 brd ff:ff:ff:ff:ff:ff
9: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ee:e9:62:47:68:9f brd ff:ff:ff:ff:ff:ff
10: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether 02:ee:97:23:56:f3 brd ff:ff:ff:ff:ff:ff
11: veth107i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr107i0 state UP group default qlen 1000
    link/ether fe:a8:67:cb:6a:0b brd ff:ff:ff:ff:ff:ff link-netnsid 0
12: fwbr107i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:44:9d:5e:7f:d3 brd ff:ff:ff:ff:ff:ff
13: fwpr107p0@fwln107i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ba:ed:b1:f9:79:b9 brd ff:ff:ff:ff:ff:ff
14: fwln107i0@fwpr107p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr107i0 state UP group default qlen 1000
    link/ether 3a:79:4d:89:ef:b5 brd ff:ff:ff:ff:ff:ff
15: veth108i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr108i0 state UP group default qlen 1000
    link/ether fe:b0:e5:da:d3:58 brd ff:ff:ff:ff:ff:ff link-netnsid 1
16: fwbr108i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 1e:cd:38:94:f6:81 brd ff:ff:ff:ff:ff:ff
17: fwpr108p0@fwln108i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 02:6e:2d:cf:81:17 brd ff:ff:ff:ff:ff:ff
18: fwln108i0@fwpr108p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr108i0 state UP group default qlen 1000
    link/ether 26:0a:34:14:2c:16 brd ff:ff:ff:ff:ff:ff

Bia (Host not having issues
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:2a:9f:97:81:2d brd ff:ff:ff:ff:ff:ff
    altname enp1s0f0
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:2a:9f:97:81:2f brd ff:ff:ff:ff:ff:ff
    altname enp1s0f1
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:2a:9f:97:81:31 brd ff:ff:ff:ff:ff:ff
    altname enp2s0f0
5: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether 00:2a:9f:97:81:33 brd ff:ff:ff:ff:ff:ff
    altname enp2s0f1
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:2a:9f:97:81:33 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.209/24 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::221:9bff:fe97:8133/64 scope link
       valid_lft forever preferred_lft forever
11: veth100i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether fe:6a:ac:0e:7d:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
12: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether be:79:d3:cb:b7:c9 brd ff:ff:ff:ff:ff:ff
13: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether a6:e7:6e:57:ab:04 brd ff:ff:ff:ff:ff:ff
14: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether 3a:83:80:95:0a:11 brd ff:ff:ff:ff:ff:ff

I just don't see any difference. It's really baffling to me.
 
Anyone have anything else I might check?

I have now set both Proxmox VE servers up to be exactly the same as far as /etc/network/interfaces (except IP Address of course), and set their /etc/resolv.conf to be the same, and ensured other than their hostname itself, that /etc/hosts and /etc/hostname is the same. I have done an `ifreload -a` on both, and still my Bia server can ping google and get to the internet with no issues, but my Aria server is only accessible from inside my network, and can only access things inside my network. Still the VMs on Aria can access the internet and external addresses, so I'm just lost.
 
No, also can't ping the router from inside the guests, but I can ping external sites like google.com and 1.1.1.1 from inside the guests
can you ping the router from the working node?

it could also just be that your router doesn't accept ICMP packets for some reason, so maybe pinging router is not the best diagnostic in your case.

I have done an `ifreload -a` on both, and still my Bia server can ping google and get to the internet with no issues, but my Aria server is only accessible from inside my network, and can only access things inside my network. Still the VMs on Aria can access the internet and external addresses, so I'm just lost.

have you checked for duplicate IPs?
maybe you can try to shutdown the node on .253 and try pinging it?

if all else fails you'll probably have to resort to using packet sniffers like tcpdump to figure out where your packets are getting lost
 
Just in case are you 100% sure of your gateway address ? i was facing the same problem as you No internet on host, all lxc container have internet access, maybe auto dhcp send the right GW addr to container idkw.

But my gw was @ .254 addr instead of .1


Second hypothesis your are on the .10 subnet, is this subnet the only on your setup ? or are you dealing with multiple subnet on your home ? in this case maybe the /24 mask is your problem
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!