The dreaded rename issue - HOWTO fix?

[iPanini]

Hi folks,

Please bear with me, here's a little background info to illustrate my situation:
A couple of years ago, must have been 2018 or 2019, I started out with Proxmox. At that time I had two Proxmox servers up and running. (Homelab situation). Somewhere 2021 we moved to a new house, and in the process all the "old" machines where powered off and stored away.
One of the machines just won't boot anymore, I suspect there's either a PSU failure or worse a MOBO problem.

In the mean time, while the elder machines were dormant, I got a free (older) machine from my employer. (Intel i5) And on that machine I have set up Proxmox as well. This machine is up and running, and named pve-1.

Now lately I've done a lot of trial and error to try and see if I could get the old machines back up and running. And with one of them this has finally worked.
So on this machine, I had Proxmox 5.5, and I think it was running Debian Stretch.
Yesterday I spent time searching and reading and managed to upgrade Debian Stretch to Debian Bullseye, and I'v also managed to update Proxmox to 7.4-18.

Now my question / issue:
After upgrading Debian and Proxmox as well and doing the renaming (following a wiki post), my pre-existing vm's show greyed out and have a question mark as status.
Search has yielded that I should copy or move the /etc/pve/nodes/pve-old-name/qemu-server/102.conf file (and subsequent other NNN.conf files) to
/etc/pve/nodes/pve-new-name/qemu-server/102.conf.
But trying to do so throws a Permission denied error.

What do I need to do to be able to move / copy these files?
I assume some running services are keeping me from moving / copying these files? Is this correct?

root@pve-0:/etc/pve/nodes/pve/qemu-server# cp 102.conf /etc/pve/nodes/pve-0/qemu-server/102.conf
cp: cannot create regular file '/etc/pve/nodes/pve-0/qemu-server/102.conf': Permission denied

root@pve-0:/etc/pve/nodes/pve/qemu-server# mv 102.conf /etc/pve/nodes/pve-0/qemu-server/102.conf
mv: cannot move '102.conf' to '/etc/pve/nodes/pve-0/qemu-server/102.conf': Permission denied

I'd like to get this solved, most of all as a learning case, so there's no vital information to be lost in this case.

Thanks for helping out!

 

[sw-omit]

You have 3 nodes in the cluster, of which only 1 is currently seen as active.
That means that you're not quorate, which in turn means that /etc/pve is locked as read-only, that's why you can't change anything.
To double-check that this is the case, run "pvecm status" and you'll see "Quorate: No" most likely (meaning less then half of the total amount of nodes are in agreement of what the current status is)
What you could do to work-around it (note, work around, not fix) would be to run "pvecm expected 1" to make it work with only 1 vote (the current server's) so you can proceed, but after that you'd need some cleaning-up most likely to be in a work-able state again.

More reading-material:
https://pve.proxmox.com/wiki/Cluster_Manager
https://forum.proxmox.com/threads/proxmox-8-config-files-only-read-only.132600/

 

[iPanini]

You have 3 nodes in the cluster, of which only 1 is currently seen as active.
That means that you're not quorate, which in turn means that /etc/pve is locked as read-only, that's why you can't change anything.
To double-check that this is the case, run "pvecm status" and you'll see "Quorate: No" most likely (meaning less then half of the total amount of nodes are in agreement of what the current status is)
What you could do to work-around it (note, work around, not fix) would be to run "pvecm expected 1" to make it work with only 1 vote (the current server's) so you can proceed, but after that you'd need some cleaning-up most likely to be in a work-able state again.

More reading-material:
https://pve.proxmox.com/wiki/Cluster_Manager
https://forum.proxmox.com/threads/proxmox-8-config-files-only-read-only.132600/

Thanks for your quick reply!
Here's the result of pvecm status:

root@pve-0:~# pvecm status
Cluster information
-------------------
Name:             pve-0
Config Version:   2
Transport:        knet
Secure auth:      on

Cannot initialize CMAP service
root@pve-0:~#

And here's the result of systemctl status pve-cluster.service

root@pve-0:~# systemctl status pve-cluster.service
● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2024-07-06 21:16:04 CEST; 16h ago
    Process: 1095 ExecStart=/usr/bin/pmxcfs (code=exited, status=0/SUCCESS)
   Main PID: 1125 (pmxcfs)
      Tasks: 7 (limit: 18978)
     Memory: 54.1M
        CPU: 40.383s
     CGroup: /system.slice/pve-cluster.service
             └─1125 /usr/bin/pmxcfs

Jul 07 13:42:35 pve-0 pmxcfs[1125]: [dcdb] crit: cpg_initialize failed: 2
Jul 07 13:42:35 pve-0 pmxcfs[1125]: [status] crit: cpg_initialize failed: 2
Jul 07 13:42:41 pve-0 pmxcfs[1125]: [quorum] crit: quorum_initialize failed: 2
Jul 07 13:42:41 pve-0 pmxcfs[1125]: [confdb] crit: cmap_initialize failed: 2
Jul 07 13:42:41 pve-0 pmxcfs[1125]: [dcdb] crit: cpg_initialize failed: 2
Jul 07 13:42:41 pve-0 pmxcfs[1125]: [status] crit: cpg_initialize failed: 2
Jul 07 13:42:47 pve-0 pmxcfs[1125]: [quorum] crit: quorum_initialize failed: 2
Jul 07 13:42:47 pve-0 pmxcfs[1125]: [confdb] crit: cmap_initialize failed: 2
Jul 07 13:42:47 pve-0 pmxcfs[1125]: [dcdb] crit: cpg_initialize failed: 2
Jul 07 13:42:47 pve-0 pmxcfs[1125]: [status] crit: cpg_initialize failed: 2

In the mean time I'll do some more reading.

 

[iPanini]

What you could do to work-around it (note, work around, not fix) would be to run "pvecm expected 1" to make it work with only 1 vote (the current server's) so you can proceed, but after that you'd need some cleaning-up most likely to be in a work-able state again.

More reading-material:
https://pve.proxmox.com/wiki/Cluster_Manager
https://forum.proxmox.com/threads/proxmox-8-config-files-only-read-only.132600/

And here's the result of running pvecm expected 1:

root@pve-0:~# pvecm expected 1
Cannot initialize CMAP service
root@pve-0:~#

 

[iPanini]

Doing some more searching I found this post.

systemctl stop pve-cluster
/usr/bin/pmxcfs -l
[main] notice: forcing local mode (althought corosync.conf exists)

I followed that after sw-omit's suggestions a couple of posts before this one,
I was then able to mv the files from the old pve name to then new pve name, then
after a reboot, I got my vm's back!

Thanks for helping out!