[SOLVED] Ghost files in PVE cluster FS

marinbernard

Active Member
Jan 2, 2019
9
1
43
38
Nice, FR
Hi,

I'm running a freshly installed & up-to-date 3-node Proxmox VE 5.3 cluster. Everything was fine until I tried to deploy custom SSL certificates for the web UI.

The process detailed in the documentation implies to add new files (pveproxy-ssl.pem & pveproxy-ssl.key) to the /etc/pve/nodes/<nodename>/ directory.

I did so on the two first nodes, and it worked fine. But on the third one, I made a mistake and misnamed the file hosting the private key: i named it pveproxy.key instead of pveproxy-ssl.key. Apparently, the PVE cluster file system did not expect such a filename, and refuses to remove it.

It is listed in ls output:
Code:
root@pve-02:/etc/pve/nodes/pve-02# ls
lrm_status  lxc  openvz  preproxy.key  priv  pve-ssl.key  pve-ssl.pem  qemu-server

but when I try to remove it, I get:

Code:
root@pve-02:/etc/pve/nodes/pve-02# rm pveproxy.key
rm: cannot remove 'pveproxy.key': No such file or directory

There's worse: I then tried to generate a new CSR with the correct file name. Again, the file is listed in ls, but I'm unable to access it.

What I tried:
  • Remove the files from another node. It failed the same way as on the main node.
  • Reboot the node. No change.
  • Stop the cluster service and mount pmxcfs locally. Still no access to the files.
I compared the versions of config.db between the 3 nodes and they seem identical (did not compare actual data, only metadata). What can I do next ? Did anyone ever experiment such a problem ?

Thanks in advance!

Marin.
 
  • Like
Reactions: marinbernard
First off, the documentation you referring to is old and not valid for PVE 5.3. See here for details. https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_certificate_management

Thanks. I'll use pvenode in the future.

And your file is named 'preproxy.key' but you try to remove 'rm pveproxy.key'.

I made a typing mistake. The command I ran had the correct file name (in fact, I used autocompletion, which does see the ghost file, while rm does not).

Any idea about how I can remove those files before I start again using pvenode ?
 
Thanks. I'll use pvenode in the future.



I made a typing mistake. The command I ran had the correct file name (in fact, I used autocompletion, which does see the ghost file, while rm does not).

Any idea about how I can remove those files before I start again using pvenode ?

Well, I tried to remove the files again, and it worked. I still don't understand why.

Thanks anyway!