3 Node cluster "permission denied - invalid PVE ticket (401)"

I solved this by setting up the same NTP server on all servers.
Yes , I also solved this by setting up the same NTP server on all servers.. today one New server I added in my cluster was having different time and date.. later as I set it manually to current time it started working. Thanks a lot, it saved my lot of time.
 
Last edited:
Helllo i was haveing the same problem the way i fixed it is :
1. Deleting this files:
<node> is your node name
  • /etc/pve/pve-root-ca.pem
  • /etc/pve/priv/pve-root-ca.key
  • /etc/pve/nodes/<node>/pve-ssl.pem
  • /etc/pve/nodes/<node>/pve-ssl.key
  • /etc/pve/authkey.pub
  • /etc/pve/priv/authkey.key
  • /etc/pve/priv/authorized_keys
2. pvecm updatecerts -f
3 systemctl restart pvedaemon pveproxy

Hope it works for others too
Your answer helped me too, thanks. ProxMox, so why is this happening? We have a 6.4-6 cluster, the problem has not been resolved yet.
 
Helllo i was haveing the same problem the way i fixed it is :
1. Deleting this files:
<node> is your node name
  • /etc/pve/pve-root-ca.pem
  • /etc/pve/priv/pve-root-ca.key
  • /etc/pve/nodes/<node>/pve-ssl.pem
  • /etc/pve/nodes/<node>/pve-ssl.key
  • /etc/pve/authkey.pub
  • /etc/pve/priv/authkey.key
  • /etc/pve/priv/authorized_keys
2. pvecm updatecerts -f
3 systemctl restart pvedaemon pveproxy

Hope it works for others too
The only solution from many that I have tried, and actually worked! Many Thanks!!!
 
So, I found my problem. For some reason, one of my nodes had wrong date, that resulted in constant 401 errors. Synchronizing the date and time, the problem went away.
 
Helllo i was haveing the same problem the way i fixed it is :
1. Deleting this files:
<node> is your node name
  • /etc/pve/pve-root-ca.pem
  • /etc/pve/priv/pve-root-ca.key
  • /etc/pve/nodes/<node>/pve-ssl.pem
  • /etc/pve/nodes/<node>/pve-ssl.key
  • /etc/pve/authkey.pub
  • /etc/pve/priv/authkey.key
  • /etc/pve/priv/authorized_keys
2. pvecm updatecerts -f
3 systemctl restart pvedaemon pveproxy

Hope it works for others too
Thanks that finally solved my issue
 
Ditto on the old thread but when I had the same issues setting up proxmox for the first time in May 2023 this thread helped point me in the right direction.

When I got the error I found one of my nodes had the wrong time.

I rebooted it, set the time locally in the bios so I could get in to fix it permanently.

On the node that had the issue I opened the Shell

Ran timedatectl it returned "System Clock Synchronization = no"

I then edited
nano /etc/chrony/chrony.conf
and under the # pool comment
changed the server named to my local NTP server

# Use Debian vendor zone
# pool 2.debian.pool.ntp.org iburst
pool 192.168.1.1 iburst

Note my local NTP server is 192.168.1.1 yours may be differant if you have one, if not pick an online NTP server that works for you.

After saving the chrony.conf file I then ran
systemctl restart chronyd

Now when I run timedatectl
it returns System Clock Sync - yes
and I don't get the error anymore.
 
I've tried all the above and thank you for the previous posters for helping to make progress. Removing the keys and doing the update certs didn't quite work for me as my system was a test homelab and probably fubarred at some stage :)

From another thread which I have now lost I got mine to work by amending /etc/pve/corosync.conf


Bash:
#totem {
#  cluster_name: homelab
#  config_version: 15
#  interface {
#    linknumber: 0
#  }
#  ip_version: ipv4-6
#  secauth: on
#  version: 2
#}

totem {
  cluster_name: homelab
  config_version: 16
  interface {
      ringnumber: 0
      knet_transport: sctp
  }
  ip_version: ipv4-6
  secauth: on
  version: 2
  token: 10000
}
 
  • Like
Reactions: elmo
So, I have gone from loving Proxmox (currently running 8.1) while it ran on a single host to hating it when trying to create a cluster and add a second host. Absolutely NOTHING I have tried to fix the 401 invalid PVE ticket error works. Yes, both servers have the same NTP server. Yes, both seem to have the exact same time. No, I can't edit corosync.conf because the file is read-only (after doing systemctl stop corosync). No, I can't delete those files mentioned in this post (pve-root-ca.pem etc.) , seems like the command just hangs (waited a few minutes). It's just for my homelab, both hosts connected to my Cisco switch (read something about unicast/multicast in another post).

Is there anyone with a step-by-step procedure to get rid of the 401 invalid PVE ticket error? Sometimes I get a connection refused 595 error too.
 
Last edited:
So, I have gone from loving Proxmox (currently running 8.1) while it ran on a single host to hating it when trying to create a cluster and add a second host. Absolutely NOTHING I have tried to fix the 401 invalid PVE ticket error works. Yes, both servers have the same NTP server. Yes, both seem to have the exact same time. No, I can't edit corosync.conf because the file is read-only (after doing systemctl stop corosync). No, I can't delete those files mentioned in this post (pve-root-ca.pem etc.) , seems like the command just hangs (waited a few minutes). It's just for my homelab, both hosts connected to my Cisco switch (read something about unicast/multicast in another post).

Is there anyone with a step-by-step procedure to get rid of the 401 invalid PVE ticket error? Sometimes I get a connection refused 595 error too.
First thing to check is to see in your cluster && corosync is working correctly.

with corosync service running on every node, what is the ouput of: "pvecm status" on each node ?
 
I broke down the cluster so now I got two separate nodes again. So far, each time I created the cluster in the GUI and joined the other host that way too. I'm trying the commandline option now and have ran pvecm create proxmox-cluster which completed succesfully, pvecm status looks good. Then, still on the same host, I ran pvecm addnode server02 and the command just seems to run forever (has been running for almost 10 minutes now). The 2nd server is showing in the GUI though but when I click it, you can guess it: connection refused (595).
 
Last edited:
This is what pvecm status shows:

Cluster information
-------------------
Name: proxmox-cluster
Config Version: 1
Transport: knet
Secure auth: on

Quorum information
------------------
Date: Sun Jan 21 15:26:18 2024
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000001
Ring ID: 1.c3
Quorate: Yes

Votequorum information
----------------------
Expected votes: 2
Highest expected: 2
Total votes: 2
Quorum: 2
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000001 1 192.168.0.199 (local)
0x00000002 1 192.168.0.200

Seems to be fine one would think, but no. Read in another post to try systemctl restart pveproxy pvestatd but that command also just seems to hang. In the same post they mention editing corosync.conf and add token: 10000. I would love to do that if I knew how I can actually save corosync.conf.
 
Last edited:
Ran into the same issue here. One of my nodes renewed its certificate and from that point on, was giving the PVE ticket error. Both nodes in my little cluster are using internal SSL certificates issued by my internal CA (not self-signed like default). They both trust the root CA certificate. Trying anything on the affected node caused it to hang pretty badly, so I powered it off. Thankfully I had replication going so getting the VMs running on the other host was pretty simple.

I'm considering just wiping and rebuilding the affected node before rejoining it to the cluster, since powering it backup will create a mess with it trying to start VMs that are now running elsewhere, etc. My concern is understanding why this happened, since I'm not keen on it happening again. I guess it's something to do with the certificate renewal but that's not much to go off
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!