2 proxmox servers, no internet access on the second one

rfdarter · Jan 19, 2025

Hi,
I added a second proxmox server to my setup and created a cluster.
I made sure that i ran `apt update && apt ugrade` on both before doing that.

I seems to work fine, my Vm`s have access to the LAN and Internet without a problem, but the host does not.

Comparing the settings of the two proxmox servers I cant seem to find a difference

`/etc/network/interfaces` on the server that has internet access:

Code:

auto lo
iface lo inet loopback

iface enp1s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.179.1/16
        gateway 192.168.178.1
        bridge-ports enp1s0
        bridge-stp off

`ip route`

Code:

default via 192.168.178.1 dev vmbr0 proto kernel onlink 
192.168.0.0/16 dev vmbr0 proto kernel scope link src 192.168.179.1

`/etc/resolv.conf`

Code:

search pi.hole
nameserver 192.168.179.2
nameserver 192.167.178.1

`/etc/hosts`

Code:

127.0.0.1 localhost.localdomain localhost
192.168.179.1 pve.fritz.box pve

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

and on the machine that does not have internet access:

Code:

auto lo
iface lo inet loopback

iface enp0s31f6 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.180.1/16
        gateway 192.168.178.1
        bridge-ports enp0s31f6
        bridge-stp off

`ip route`

Code:

default via 192.168.178.1 dev vmbr0 proto kernel onlink 
192.168.0.0/16 dev vmbr0 proto kernel scope link src 192.168.180.1

`/etc/resolv.conf`

Code:

search pi.hole
nameserver 192.168.179.2
nameserver 192.168.178.1

`/etc/hosts`

Code:

127.0.0.1 localhost.localdomain localhost
192.168.180.1 pve2.fritz.box pve2

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

My router recognizes the seconds server fine and displays the right ip address.
`nslookup` works alsow fine and i get an ip from pihole but a ping fails

Code:

root@pve2:~# nslookup google.com
Server:         192.168.179.2
Address:        192.168.179.2#53

Non-authoritative answer:
Name:   google.com
Address: 142.251.36.174
Name:   google.com
Address: 2a00:1450:4016:808::200e

root@pve2:~# ping google.com
PING google.com (142.251.36.174) 56(84) bytes of data.
From pve2.fritz.box (192.168.180.1) icmp_seq=1 Destination Host Unreachable

I hope someone can help me.

UdoB · Jan 19, 2025

rfdarter said:
I made sure that i ran `apt update && apt ugrade` on both before doing that.

That may have been bad. Always use "full-upgrade" or "dist-upgrade", see: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#system_software_updates

Some upgrade in the past changed the NIC names. Search for it here in the forum.

More hints: https://forum.proxmox.com/threads/f...can-not-load-the-web-gui-in-a-browser.160091/

rfdarter · Jan 19, 2025

Thanks, I followed a yt video which stated you should make sure they are both on the same software version, so in my mind i never even thought about somthing else than `apt update && apt upgrade`

But since i never ran "full-upgrade" or "dist-upgrade" on any of the machines and I installed both form the same ISO i think it should be fine?

I looked at the posted tutorial link, but the wiring should be fine since the VM`s can access the internet.
The configurations looks also good or do you see a mistake I made?

UdoB · Jan 19, 2025

rfdarter said:
But since i never ran "full-upgrade" or "dist-upgrade" on any of the machines and I installed both form the same ISO i think it should be fine?

Yes, I think it is fine. Just run "apt full-upgrade" now and in future.

rfdarter said:
I looked at the posted tutorial link, but the wiring should be fine since the VM`s can access the internet.

Okay, that article was not meant to fit exactly to your situation but is more general.

rfdarter said:
The configurations looks also good or do you see a mistake I made?

No, everything I see looks fine. But it lacks a simple ip addr show...

And... a "/16" for 192.168 is really unusual - while technically ok. Are you sure the router does accept that? For example an "AVM Fritz!Box" uses 192.168.178.0/24 by default (as far as I remember)... and your "192.168.180.1" would not be inside that network...

rfdarter · Jan 19, 2025

UdoB said:
ip addr show

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether 90:1b:0e:da:0c:17 brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 90:1b:0e:da:0c:17 brd ff:ff:ff:ff:ff:ff
    inet 192.168.180.1/16 scope global vmbr0
       valid_lft forever preferred_lft forever
4: veth108i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr108i0 state UP group default qlen 1000
    link/ether fe:3b:f8:ad:2a:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0
5: fwbr108i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 1e:49:e8:92:b0:0e brd ff:ff:ff:ff:ff:ff
6: fwpr108p0@fwln108i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 3a:e4:42:89:46:ea brd ff:ff:ff:ff:ff:ff
7: fwln108i0@fwpr108p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr108i0 state UP group default qlen 1000
    link/ether 1e:49:e8:92:b0:0e brd ff:ff:ff:ff:ff:ff
8: veth200i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:e7:ac:a7:5d:71 brd ff:ff:ff:ff:ff:ff link-netnsid 1
13: veth210i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr210i0 state UP group default qlen 1000
    link/ether fe:3b:e2:74:4c:06 brd ff:ff:ff:ff:ff:ff link-netnsid 2
14: fwbr210i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 26:79:2b:d9:a7:1c brd ff:ff:ff:ff:ff:ff
15: fwpr210p0@fwln210i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 82:a3:ab:0a:78:77 brd ff:ff:ff:ff:ff:ff
16: fwln210i0@fwpr210p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr210i0 state UP group default qlen 1000
    link/ether 26:79:2b:d9:a7:1c brd ff:ff:ff:ff:ff:ff

I used '/16' on purpose to give each proxmox server a base ip adress of `192.168.179.1` and `192.168.180.1` and the VM`s `192.168.179.2` to `192.168.179.254` and `192.168.180.2` to `192.168.180.254`
So i could easily tell by the ip on which proxmox server the Vm is running.
Not a good idea? Is a `/16` in general not a good idea or just using `192.168`?

Yes on a FrizBox the default is 192.168.178.0/24 but i have set it to 192.168.0.0/16
My Proxmox server using the ip `192.168.179.1` works fine, and can access the internet, just the on using `192.168.180.1` can not

rfdarter · Jan 19, 2025

I just noticed something.
From the proxmox server that can not connect to the internet i can ping everything in the LAN except the gateway(192.168.178.1)

UdoB · Jan 20, 2025

I still do not see anything wrong, sorry.

rfdarter said:
I used '/16' on purpose to give each proxmox server a base ip adress of `192.168.179.1` and `192.168.180.1` and the VM`s `192.168.179.2` to `192.168.179.254` and `192.168.180.2` to `192.168.180.254`
So i could easily tell by the ip on which proxmox server the Vm is running.
Not a good idea? Is a `/16` in general not a good idea or just using `192.168`?

It is technically fine and it (should) work flawlessly.

You can put 65534 computers into that network. Do you plan to use so many? Or would it be more probable to have some much smaller groups of devices, in networks with less than 254 computers --> /24?

Only with separate networks you can introduce a router to separate traffic between those groups of devices. And there may be several of them: wan, dmz, media, wlan-private, wlan-guests, wlan-iot, storage, admin, isolated, tor, backup, etc. This is the basis for to be able to forbid an IoT lightbulb from China or an Amazon owned camera or any Android phone to scan your private samba share on all computers in your home.

All of this is not your current problem though...

DBS · Jan 20, 2025

Hi Udo, I do have a similar problem but more weird.

Situation: 1 cluster with 3 nodes and 2 stand-alone nodes.
Cluster is running 8.0.2
Stand-alone is running 8.2.2

Al servers are in the same subnet and do have the same gateway.

The Cluster do have internet access.
Stand-alone don't have internet access.

The VM's however are working fine.

So fyi, i'm working on it, but priority is a little low.
Maybe an clean install will solve it

Regards,
Ronald

UdoB · Jan 20, 2025

DBS said:
Hi Udo, I do have a similar problem but more weird.

I recommend to open a new thread for new problems: while they look similar your setup is probably different from @rfdarter's

DBS said:
Situation: 1 cluster with 3 nodes and 2 stand-alone nodes.

Al servers are in the same subnet and do have the same gateway.

The Cluster do have internet access.
Stand-alone don't have internet access.

Well, you need to post some more information - my usual reply: you gave us zero information about your setup. The more details you post here, the greater the chance for a helpful answer

Please start by giving us some information, like the copy-n-pasted output of some commands, run on a PVE host (either via SSH or via "Datacenter --> <one Node> --> Shell", both ways allow copy-n-paste):

PVE System information:

pveversion -v

Basic network information:

ip address show # currently active IP addresses on one NODE
ip route show # currently active routing table on one NODE
ip link show # currently active links on one NODE
cat /etc/network/interfaces # configuration of the network
cat /etc/resolv.conf # DNS resolver settings
ping -c 1 -W 1 9.9.9.9 # a simple "ping" to verify outgoing routing
host quad9.net # a simple DNS lookup to verify reachable DNS servers

Those are examples. You may add/edit commands and options if you can enrich the information given. Oh, and please put each command in a separate [CODE]...[/CODE]-block for better readability.

Please repeat the above for one node in the cluster and for one of those stand-alone servers - to give us a chance to compare both ;-)

DBS · Jan 20, 2025

Hi Udo, it was just a f.y.i. to mention it is happening more.
No help needed for this one ;-)

Regards,
Ronald

Search

Search

2 proxmox servers, no internet access on the second one

rfdarter

New Member

UdoB

Distinguished Member

rfdarter

New Member

UdoB

Distinguished Member

rfdarter

New Member

rfdarter

New Member

UdoB

Distinguished Member

DBS

Renowned Member

UdoB

Distinguished Member

DBS

Renowned Member

We value your privacy