Wrong IP shown joining cluster - local node address: cannot use IP not found on local node!

harmonyp · Jan 26, 2021

When trying to join through the GUI or SSH I am faced with this error. 192.168.0.50 & 192.168.0.201 can ping eachother

Code:

detected the following error(s):
* local node address: cannot use IP '0.48.150.92', not found on local node!


TASK ERROR: Check if node may join a cluster failed!

As you can see the join IP is shown incorrectly below

root@3173716:~# pvecm status

Code:

Cluster information
-------------------
Name:             france
Config Version:   1
Transport:        knet
Secure auth:      on


Quorum information
------------------
Date:             Tue Jan 26 13:04:54 2021
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000001
Ring ID:          1.23
Quorate:          Yes


Votequorum information
----------------------
Expected votes:   1
Highest expected: 1
Total votes:      1
Quorum:           1
Flags:            Quorate


Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 192.168.0.50 (local)

root@3173716:~# ip address

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether d0:50:99:d8:8e:df brd ff:ff:ff:ff:ff:ff
259: tap406i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
    link/ether 1e:21:28:67:25:1e brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether d0:50:99:d8:8e:de brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.50/16 scope global eno2
       valid_lft forever preferred_lft forever
    inet6 fe80::d250:99ff:fed8:8ede/64 scope link
       valid_lft forever preferred_lft forever
4: enp0s20f0u8u3c2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:06:b0:b9:1f:cd brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000

/etc/network/interfaces

Code:

auto lo
iface lo inet loopback


auto eno2
iface eno2 inet static
        address 192.168.0.50/16


iface eno1 inet manual


iface enp0s20f0u8u3c2 inet manual


auto vmbr0
iface vmbr0 inet static
        address 51.210.99.xxx/24
        gateway 51.210.99.254
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0


iface vmbr0 inet6 static
        address 2001:41d0:303:xxxx::1/128
        gateway 2001:41d0:303:d9ff:ff:ff:ff:ff
        post-up /sbin/ip -f inet6 route add 2001:41d0:303:d96ff:ff:ff:ff:ff dev vmbr0
        post-up /sbin/ip -f inet6 route add default via 2001:41d0:303:d96ff:ff:ff:ff:ff
        pre-down /sbin/ip -f inet6 route del default via 2001:41d0:303:d96ff:ff:ff:ff:ff200
        pre-down /sbin/ip -f inet6 route del 2001:41d0:303:d96ff:ff:ff:ff:ff dev vmbr0




/etc/hosts


127.0.0.1       localhost
192.168.0.50 3173716.mydomain.com 3173716
192.168.0.201 3184220.mydomain.com 3184220
51.210.99.xxx 3173716.mydomain.com 3173716


::1     localhost       ip6-localhost   ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

/etc/pve/corosync.conf

Code:

logging {
  debug: off
  to_syslog: yes
}


nodelist {
  node {
    name: 3173716
    nodeid: 1
    quorum_votes: 1
    ring0_addr: 192.168.0.50
  }
}


quorum {
  provider: corosync_votequorum
}


totem {
  cluster_name: france
  config_version: 1
  interface {
    linknumber: 0
  }
  ip_version: ipv4-6
  link_mode: passive
  secauth: on
  version: 2
}

On my second node trying to join:

root@3184220:~# ip address

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defaul                                                                                                                                                             t qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 st                                                                                                                                                             ate UP group default qlen 1000
    link/ether a8:a1:59:36:54:42 brd ff:ff:ff:ff:ff:ff
3: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group                                                                                                                                                              default qlen 1000
    link/ether a8:a1:59:36:60:ce brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.201/16 brd 192.168.255.255 scope global enp1s0f1
       valid_lft forever preferred_lft forever
    inet6 fe80::aaa1:59ff:fe36:60ce/64 scope link
       valid_lft forever preferred_lft forever
4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP grou                                                                                                                                                             p default qlen 1000
    link/ether a8:a1:59:36:54:42 brd ff:ff:ff:ff:ff:ff
    inet 135.125.2.xxx/24 brd 135.125.2.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 2001:41d0:203:966e::1/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::aaa1:59ff:fe36:5442/64 scope link
       valid_lft forever preferred_lft forever

/etc/hosts/

Code:

127.0.0.1       localhost.localdomain   localhost
135.125.2.xxx   3184220.mydomain.com   3184220
192.168.0.50 3173716.mydomain.com 3173716
192.168.0.201 3184220.mydomain.com 3184220


# The following lines are desirable for IPv6 capable hosts
::1     localhost       ip6-localhost   ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

harmonyp · Jan 27, 2021

Bump, anyone?

dcsapak · Jan 27, 2021

mhmm... i have not really a solution for now (it seems to be a bug) but i can see the reason:

harmonyp said:
192.168.0.50 3173716.mydomain.com 3173716

3173716 interpreted as an ip is... 0.48.109.84

so somewhere in the stack the hostname '3173716' gets interpreted as an ipv4 adress...

harmonyp · Jan 27, 2021

dcsapak said:
mhmm... i have not really a solution for now (it seems to be a bug) but i can see the reason:

3173716 interpreted as an ip is... 0.48.109.84

so somewhere in the stack the hostname '3173716' gets interpreted as an ipv4 adress...

If it helps I installed Proxmox from Debian10 following https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster

root@3173716:~# hostname --ip-address
51.210.99.xxx

One thing I have noticed it has /etc/hosts keeps changing when I reboot the server I had to lock the file using

chattr +i /etc/hosts

The following gets added to /etc/hosts

Code:

127.0.1.1       ns3173716.xyz       ns3173716

It's the rDNS record.

Pierre-Yves · Jan 27, 2021

Hi, it seems like a problem I've encountered ...
Try to switch the configuration between en1 and en2.
en1 will have 192.168.0.50/16 and en2 will have your bridge
the first nic shoud be the one that is used to operate the cluster
it solves the same problem for me in the past
Regards

harmonyp · Jan 27, 2021

Pierre-Yves said:
Hi, it seems like a problem I've encountered ...
Try to switch the configuration between en1 and en2.
en1 will have 192.168.0.50/16 and en2 will have your bridge
the first nic shoud be the one that is used to operate the cluster
it solves the same problem for me in the past
Regards

That broke the network. It's an OVH server d0:50:99:d8:8e:df is the public network interface and d0:50:99:d8:8e:de is the vRACK.

Pierre-Yves · Jan 27, 2021

Does the ping runs with the names of the servers, not the IPs (ping 3173716) ?

harmonyp · Jan 27, 2021

No it doesn't

Code:

root@3173716:~# ping 3173716
PING 3173716 (0.48.109.84) 56(84) bytes of data.

While accessing the server through IPMI I noticed the login screen shows the address too

"Welcome to the Proxmox Virtual Envionment. Please use your web browser to configure this server - connect to:

https://0.48.150.92:8006"

Pierre-Yves · Jan 27, 2021

Thats a problem !
https://vitux.com/linux-hosts-file/

As long as you cannot resolve by name in your cluster, you will have trouble ...

Regards

harmonyp · Jan 27, 2021

Pierre-Yves said:
Thats a problem !
https://vitux.com/linux-hosts-file/

As long as you cannot resolve by name in your cluster, you will have trouble ...

Regards

But it should be working? Is it because it's numbers only?

Pierre-Yves · Jan 27, 2021

type "more /etc/hostname" to know the name of your host

check that you have in your /etc/hosts file things like
1.2.3.4 name_of_your_host
1.2.3.5 name_of_your_second_host
etc ...

or you can edit it thru the Proxmox GUI by selecting your server in the left column then system / hosts
in this file you define what name is associated with a specific IP

thanks to this system, your computer can find IP from a name

all the proxmox hostss must know all the ip / name of all the proxmox hosts in the cluster to communicate properly

Regards

Pierre-Yves · Jan 27, 2021

I'm thinking about a important point : is the proxmox firewall activated ?

harmonyp · Jan 27, 2021

Pierre-Yves said:
type "more /etc/hostname" to know the name of your host

check that you have in your /etc/hosts file things like
1.2.3.4 name_of_your_host
1.2.3.5 name_of_your_second_host
etc ...

or you can edit it thru the Proxmox GUI by selecting your server in the left column then system / hosts
in this file you define what name is associated with a specific IP

thanks to this system, your computer can find IP from a name

all the proxmox hostss must know all the ip / name of all the proxmox hosts in the cluster to communicate properly

Regards

Thanks very much for your help.

Changing the hostname from 3173716.mydomain.com to ns3173716.mydomain.com and updating

51.210.99.xxx 3173716.mydomain.com 3173716

to

51.210.99.xxx 3173716mydomain.com ns3173716

has fixed the issue. Also no firewall.

fiona · Feb 2, 2021

Hi,

dcsapak said:
mhmm... i have not really a solution for now (it seems to be a bug) but i can see the reason:

3173716 interpreted as an ip is... 0.48.109.84

so somewhere in the stack the hostname '3173716' gets interpreted as an ipv4 adress...

It's all the way down in glibc, and it's probably best to avoid all-numeric hostnames (for now).

Search

Search

Wrong IP shown joining cluster - local node address: cannot use IP not found on local node!

harmonyp

Member

harmonyp

Member

dcsapak

Proxmox Staff Member

harmonyp

Member

Pierre-Yves

Member

harmonyp

Member

Pierre-Yves

Member

harmonyp

Member

Pierre-Yves

Member

harmonyp

Member

Pierre-Yves

Member

Pierre-Yves

Member

harmonyp

Member

fiona

Proxmox Staff Member