Failed to add cluster node "401 No ticket"

n1nj4888

Well-Known Member
Jan 13, 2019
162
22
58
44
Can anyone help with the following error I'm getting after creating a cluster on a single node and attempting to add a second node to the cluster?

I've created a cluster on the first PVE Node (call it NODE1) which has the following networks configured:

DefaultLAN - 192.168.1.NODE1
ClusterHB (Ring0 Address) - 192.168.2.NODE1

I also have a second standalone PVE Node (call it NODE2) which has the following networks configured:

DefaultLAN - 192.168.1.NODE2
ClusterHB (Ring0 Address) - 192.168.2.NODE2

When i copy the Cluster Join Information from NODE1 to NODE2 (in the GUI of NODE2), I paste the Join Information into "Join Cluster" and:

1) Peer Address is auto-populated with the DefaultLAN address of NODE1 (i.e. 192.168.1.NODE1 not the NODE1 Ring0 Address - Assume this correct?)
2) Corosync Ring 0 is populated with a warning: "Cannot use default address safely" so I update this to be ClusterHB (Ring 0 Address) of NODE2 i.e. 192.168.2.NODE2
3) Enter the root password of NODE1
4) Click "Join Cluster"

The following error is displayed in the Task Viewer for the "Join Cluster" job?
Establishing API connection with host '192.168.1.NODE1'
Login succeeded.
Request addition of this node
TASK ERROR: 401 No ticket


I also get the same type of error if I try to add NODE2 to the cluster via NODE2 console:

root@NODE2:~# pvecm add 192.168.1.NODE1 -ring0_addr 192.168.2.NODE2
Please enter superuser (root) password for '192.168.1.NODE1':
Password for root@192.168.1.NODE1: ***
Establishing API connection with host '192.168.1.NODE1'
The authenticity of host '192.168.1.NODE1' can't be established.
X509 SHA256 key fingerprint is XXX.
Are you sure you want to continue connecting (yes/no)? yes
Login succeeded.
Request addition of this node
401 No ticket
 
Can you please post the cluster corosync configuration from the node you created the cluster? E.g.:
Code:
cat /etc/pve/corosync.conf
It would be great to have uncensored, as long as those node use private IPv4 address ranges, like 192.168.0.0/16 or 10.0.0.0/8, only.

Can you login over the web interface as root on both nodes normally? What's going on in the syslog/journal during this error?

Also, as workaround - joining over SSH is still available, you could try it by adding the "--use_ssh" to the pvecm add commando.
 
Thanks for the reply!

Here is /etc/pve/corosync.conf from NODE1 (where I created the cluster)

Code:
logging {
  debug: off
  to_syslog: yes
}

nodelist {
  node {
    name: pve-host1
    nodeid: 1
    quorum_votes: 1
    ring0_addr: 192.168.2.201
  }
}

quorum {
  provider: corosync_votequorum
}

totem {
  cluster_name: pve-cluster1
  config_version: 1
  interface {
    bindnetaddr: 192.168.2.201
    ringnumber: 0
  }
  ip_version: ipv4
  secauth: on
  version: 2
}

Yes - I can login to the WebUI as root on both NODE1 and NODE2 as expected...

The following is the output on NODE2 syslog when attempting to join the cluster through the NODE2 WebUI:

Code:
Jun  4 12:30:09 pve-host2.local pvedaemon[29538]: <root@pam> starting task UPID:pve-host2:<ID>:clusterjoin::root@pam:
Jun  4 12:30:12 pve-host2.local pvedaemon[14119]: 401 No ticket
Jun  4 12:30:12 pve-host2.local pvedaemon[29538]: <root@pam> end task UPID:pve-host2:<ID>:clusterjoin::root@pam: 401 No ticket

There doesn't seem to be anything of interest in the NODE1 syslog at the point of attempting to join the cluster from NODE2

 
I didn't try the pvecm "--use_ssh" command but, I've found the cause of the issue...

The reason for the error is seemingly because 2FA was turned on for root on both NODE1 and NODE2. As soon as I disabled 2FA (on both nodes) and repasted the Join Information into NODE2, NODE2 was added to the cluster successfully...

2FA is not mentioned anywhere as being a potential issue in the Cluster Manager wiki (https://pve.proxmox.com/wiki/Cluster_Manager) and it's not immediately clear that this could be an issue when adding a node to the cluster so it would be good to either (A) update the wiki to point out that 2FA needs to be disabled for this to work or (B) allow the user to enter the NODE1 2FA key in the NODE2 Join Cluster dialog when attempting to join NODE2 to the cluster

Thanks!

EDIT: After removing 2FA from both nodes and adding NODE2 to the cluster, I was able to re-enable 2FA on NODE1 and confirmed that the same 2FA key can be used to login to either NODE1 or NODE2
 
Last edited:
The reason for the error is seemingly because 2FA was turned on for root on both NODE1 and NODE2. As soon as I disabled 2FA (on both nodes) and repasted the Join Information into NODE2, NODE2 was added to the cluster successfully...
Ahh, yeah, now the error makes sense..

2FA is not mentioned anywhere as being a potential issue in the Cluster Manager wiki (https://pve.proxmox.com/wiki/Cluster_Manager) and it's not immediately clear that this could be an issue when adding a node to the cluster so it would be good to either (A) update the wiki to point out that 2FA needs to be disabled for this to work or (B) allow the user to enter the NODE1 2FA key in the NODE2 Join Cluster dialog when attempting to join NODE2 to the cluster

Yes, there we definitively lack some information... The cluster join API wasn't adapted to the new improved TFA integration...
So, for TOTP we could really allow to enter the code if it's set, for U2F it's not to easy to interact with the keys in a manageable way, not sure about that one, here we could at least warn the user on the CLI, for webUI it could be even done...
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!