Issues after renaming node

instantautomation

New Member
Sep 19, 2021
9
0
1
30
I followed the instructions here https://pve.proxmox.com/wiki/Renaming_a_PVE_node and the name changed but all of my container/VM configs were not there. After checking into /etc/pve/nodes the old node was there along with the new. I moved my configs into the new nodes folder, verified they were there and rebooted. Now I think I broke some stuff because I cannot connect to the gui and I cannot update certs. I get an error saying the directory doesn't exist. Also the /etc/pve/nodes doesn't exist either. I can connect via ssh.
 
Last edited:
Hi,
please share the output of the following:
Code:
cat /etc/hostname
cat /etc/hosts
systemctl pve-cluster.service
pveversion -v
 
Thanks for your reply,

Info as requested.

root@Creator:~# cat /etc/hostname
Creator

root@Creator:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
192.168.88.10 Creator.Proxmox.Server Creator

# The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

root@Creator:~# systemctl pve-cluster.service
Unknown command verb pve-cluster.service.

root@Creator:~# pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-4-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.4: 6.4-6
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-11
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-3
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.5-1
pve-qemu-kvm: 6.0.0-4
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1
 
Thanks for your reply,

Info as requested.

root@Creator:~# cat /etc/hostname
Creator

root@Creator:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
192.168.88.10 Creator.Proxmox.Server Creator
Is the IP actively configured on the host, i.e. shows up in the output of ip addr?

# The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

root@Creator:~# systemctl pve-cluster.service
Unknown command verb pve-cluster.service.
Sorry, I meant systemctl status pve-cluster.service. If there's an error journalctl -b0 -u pve-cluster.service might show a bit more.

root@Creator:~# pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-4-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.4: 6.4-6
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.4.140-1-pve: 5.4.140-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-11
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-3
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.5-1
pve-qemu-kvm: 6.0.0-4
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1
 
root@Creator:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
link/ether b4:2e:99:ec:5a:9c brd ff:ff:ff:ff:ff:ff
altname enp4s0
3: enp6s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr1 state DOWN group default qlen 1000
link/ether a0:36:9f:8a:6b:e4 brd ff:ff:ff:ff:ff:ff
4: enp6s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr2 state DOWN group default qlen 1000
link/ether a0:36:9f:8a:6b:e5 brd ff:ff:ff:ff:ff:ff
5: enp6s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:8a:6b:e6 brd ff:ff:ff:ff:ff:ff
6: enp6s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether a0:36:9f:8a:6b:e7 brd ff:ff:ff:ff:ff:ff
7: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 12:d2:a2:10:2b:45 brd ff:ff:ff:ff:ff:ff
inet 192.168.88.10/24 brd 192.168.88.255 scope global vmbr0
valid_lft forever preferred_lft forever
inet6 fe80::10d2:a2ff:fe10:2b45/64 scope link
valid_lft forever preferred_lft forever
8: vmbr1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 86:f3:02:21:61:3e brd ff:ff:ff:ff:ff:ff
9: vmbr2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether a2:4f:8b:9d:ae:09 brd ff:ff:ff:ff:ff:ff


root@Creator:~# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-09-20 06:03:23 MST; 12h ago
Process: 5845 ExecStart=/usr/bin/pmxcfs (code=exited, status=255/EXCEPTION)
CPU: 6ms

Sep 20 06:03:23 Creator systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 5.
Sep 20 06:03:23 Creator systemd[1]: Stopped The Proxmox VE cluster filesystem.
Sep 20 06:03:23 Creator systemd[1]: pve-cluster.service: Start request repeated too quickly.
Sep 20 06:03:23 Creator systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Sep 20 06:03:23 Creator systemd[1]: Failed to start The Proxmox VE cluster filesystem.
 
root@Creator:~# journalctl -b0 -u pve-cluster.service
-- Journal begins at Sun 2021-09-19 13:45:27 MST, ends at Wed 2021-09-22 04:49:57 MST. --
Sep 20 04:30:26 Creator systemd[1]: Starting The Proxmox VE cluster filesystem...
Sep 20 04:30:26 Creator pmxcfs[1757]: [database] crit: found entry with duplicate name 'lxc' - A:(inode = 0x00000000007CCDC7, parent = 0x00000000007C>
Sep 20 04:30:26 Creator pmxcfs[1757]: [database] crit: DB load failed
Sep 20 04:30:26 Creator pmxcfs[1757]: [database] crit: found entry with duplicate name 'lxc' - A:(inode = 0x00000000007CCDC7, parent = 0x00000000007C>
Sep 20 04:30:26 Creator pmxcfs[1757]: [database] crit: DB load failed
Sep 20 04:30:26 Creator pmxcfs[1757]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Sep 20 04:30:26 Creator pmxcfs[1757]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 20 04:30:26 Creator pmxcfs[1757]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Sep 20 04:30:26 Creator pmxcfs[1757]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 20 04:30:26 Creator systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Sep 20 04:30:26 Creator systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Sep 20 04:30:26 Creator systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Sep 20 04:30:27 Creator systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 1.
Sep 20 04:30:27 Creator systemd[1]: Stopped The Proxmox VE cluster filesystem.
Sep 20 04:30:27 Creator systemd[1]: Starting The Proxmox VE cluster filesystem...
Sep 20 04:30:27 Creator pmxcfs[1905]: [database] crit: found entry with duplicate name 'lxc' - A:(inode = 0x00000000007CCDC7, parent = 0x00000000007C>
Sep 20 04:30:27 Creator pmxcfs[1905]: [database] crit: found entry with duplicate name 'lxc' - A:(inode = 0x00000000007CCDC7, parent = 0x00000000007C>
Sep 20 04:30:27 Creator pmxcfs[1905]: [database] crit: DB load failed
Sep 20 04:30:27 Creator pmxcfs[1905]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Sep 20 04:30:27 Creator pmxcfs[1905]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 20 04:30:27 Creator pmxcfs[1905]: [database] crit: DB load failed
Sep 20 04:30:27 Creator pmxcfs[1905]: [main] crit: memdb_open failed - unable to open database '/var/lib/pve-cluster/config.db'
Sep 20 04:30:27 Creator pmxcfs[1905]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 20 04:30:27 Creator systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Sep 20 04:30:27 Creator systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Sep 20 04:30:27 Creator systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Sep 20 04:30:27 Creator systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 2.
Sep 20 04:30:27 Creator systemd[1]: Stopped The Proxmox VE cluster filesystem.
Sep 20 04:30:27 Creator systemd[1]: Starting The Proxmox VE cluster filesystem...
Sep 20 04:30:27 Creator pmxcfs[1908]: [database] crit: found entry with duplicate name 'lxc' - A:(inode = 0x00000000007CCDC7, parent = 0x00000000007C>
 
Please follow the steps described in this thread closely (the later half starting from the post the link points to). If there are questions, feel free to ask.
 
Thanks for the help,
I ran into this error after trying to start the pve-cluster


Sep 24 09:05:25 Creator pveproxy[1919]: worker 256127 finished
Sep 24 09:05:25 Creator pveproxy[1919]: starting 1 worker(s)
Sep 24 09:05:25 Creator pveproxy[1919]: worker 256130 started
Sep 24 09:05:25 Creator pveproxy[256130]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1891.

There are several line of the same error.

I think this is where my problem is [database] crit: found entry with duplicate name 'lxc'
Although I am unsure of where to look to remove the duplicate.
 
Last edited:
BTW I do not get an output of this command

sqlite3 /var/lib/pve-cluster/config.db 'SELECT inode,parent,mtime,type,name FROM tree WHERE parent = (SELECT parent FROM tree WHERE inode = 12430028)'

And this is the output of the first

root@Creator:~# sqlite3 /var/lib/pve-cluster/config.db 'SELECT inode,parent,mtime,type,name FROM tree WHERE name = "authorized_keys"'
8179152|9|1632074413|8|authorized_keys
 
Last edited:
I think this is where my problem is [database] crit: found entry with duplicate name 'lxc'

Although I am unsure of where to look to remove the duplicate.
Yes, to fix the database it's hopefully enough to remove that duplicate.

BTW I do not get an output of this command

sqlite3 /var/lib/pve-cluster/config.db 'SELECT inode,parent,mtime,type,name FROM tree WHERE parent = (SELECT parent FROM tree WHERE inode = 12430028)'
In your case the inode is 0x00000000007CCDC7 (can be seen in your log) which can be converted to decimal with e.g.
Code:
echo "obase=10; ibase=16; 00000000007CCDC7" | bc
and that gives 8179143, so you need to use that instead of 12430028.

And this is the output of the first

root@Creator:~# sqlite3 /var/lib/pve-cluster/config.db 'SELECT inode,parent,mtime,type,name FROM tree WHERE name = "authorized_keys"'
8179152|9|1632074413|8|authorized_keys
Here you need to use lxc instead of authorized_keys, because that's the duplicate entry causing problems.
 
Alright after making those changes here is the error I'm getting

-- Journal begins at Sun 2021-09-19 13:45:27 MST, ends at Wed 2021-09-29 10:24:35 MST. --
Sep 29 10:21:48 Creator systemd[1]: Starting The Proxmox VE cluster filesystem...
Sep 29 10:21:48 Creator pmxcfs[1418]: fuse: mountpoint is not empty
Sep 29 10:21:48 Creator pmxcfs[1418]: fuse: if you are sure this is safe, use the 'nonempty' mount option
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] crit: fuse_mount error: File exists
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] crit: fuse_mount error: File exists
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 29 10:21:48 Creator systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Sep 29 10:21:48 Creator systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Sep 29 10:21:48 Creator systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Sep 29 10:21:48 Creator systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 1.
Sep 29 10:21:48 Creator systemd[1]: Stopped The Proxmox VE cluster filesystem.
 
Alright after making those changes here is the error I'm getting

-- Journal begins at Sun 2021-09-19 13:45:27 MST, ends at Wed 2021-09-29 10:24:35 MST. --
Sep 29 10:21:48 Creator systemd[1]: Starting The Proxmox VE cluster filesystem...
Sep 29 10:21:48 Creator pmxcfs[1418]: fuse: mountpoint is not empty
Sounds like something has been written to /etc/pve while the cluster filesystem was not mounted there. Try moving those files somewhere else (to inspect them) and try starting the service again.

Sep 29 10:21:48 Creator pmxcfs[1418]: fuse: if you are sure this is safe, use the 'nonempty' mount option
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] crit: fuse_mount error: File exists
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] crit: fuse_mount error: File exists
Sep 29 10:21:48 Creator pmxcfs[1418]: [main] notice: exit proxmox configuration filesystem (-1)
Sep 29 10:21:48 Creator systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION
Sep 29 10:21:48 Creator systemd[1]: pve-cluster.service: Failed with result 'exit-code'.
Sep 29 10:21:48 Creator systemd[1]: Failed to start The Proxmox VE cluster filesystem.
Sep 29 10:21:48 Creator systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 1.
Sep 29 10:21:48 Creator systemd[1]: Stopped The Proxmox VE cluster filesystem.
 
So I have tried moving all files aside from the SSL key stuff and I am still getting the same error. Do you have any other suggestions?
 
So I have tried moving all files aside from the SSL key stuff and I am still getting the same error. Do you have any other suggestions?
The mount point /etc/pve has to be completely empty. Just move the files back (if they are not already present) after the cluster file system has been mounted there.
 
Took me a while to get back to this. Moving everything allowed the file system to start. now I just need to restore my configs. Thank you for all your help!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!