[TUTORIAL] [Recovery] Mount pmxcfs offline, extract configurations - /etc/pve

[esi_y]

There are cases when you make changes in your configurations, only to want to partially revert it back.

NOTE See how you should have already been backing it up [1].

Alternatively, you get hold of stale (from non quorate-node) or partially corrupt config.db - see also how to recover it [1] - and want to take out only some of the previous files. without making it your current node's cluster filesystem.

Less often, you might want to edit the contents of the database-backed filesystem without side effects to the node or cluster, e.g. in order to implant it into a separate/cloned/new cluster.

DISCLAIMER If you do not understand the summary above, do NOT proceed.

This is actually possible, however since the pmxcfs [2] relies on hardcoded locations for its backend database file as well as mountpoint, you would need to use chroot [3].

mkdir -p ~/jail-pmxcfs/{dev,usr,bin,sbin,lib,lib64,etc,var/lib/pve-cluster,var/run}
for i in /dev /usr /bin /sbin /lib /lib64 /etc; do mount --bind -o ro $i /root/jail-pmxcfs/$i; done

This will create alternative root structure for your own instance of pmxcfs, the only thing left is to implant the database of interest, in this example from existing one:

sqlite3 /var/lib/pve-cluster/config.db .dump > ~/config.dump.sql
sqlite3 ~/jail-pmxcfs/var/lib/pve-cluster/config.db < ~/config.dump.sql

Now launch your own pmxcfs instance in local mode (-l) in the chroot environment:

chroot ~/jail-pmxcfs/ pmxcfs -l

You can double check your instance runs using the database file that was just provided:

# lsof ~/jail-pmxcfs/var/lib/pve-cluster/config.db

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
pmxcfs  1225 root    4u   REG  252,1    77824   61 /root/jail-pmxcfs/var/lib/pve-cluster/config.db

In fact, if you have the regular pve-cluster service running, you will be able to see there's two instances running, each over its own database, the new one in local mode (-l):

# ps -C pmxcfs -f

UID          PID    PPID  C STIME TTY          TIME CMD
root         656       1  0 10:34 ?        00:00:02 /usr/bin/pmxcfs
root        1225       1  0 10:37 ?        00:00:00 pmxcfs -l

Now you can copy out your files or perform changes in ~/jail-pmxcfs/etc/pve without affecting your regular operation.

You can also make an SQL dump [1] of ~/jail-pmxcfs/var/lib/pve-cluster/config.db - your now modified database.

Once you are finished, you will want to get rid of the extra instance (based on the PID of the local (-l) instance obtained above):

kill $PID

And destroy the temporary chroot structure:

umount ~/jail-pmxcfs/etc/pve ~/jail-pmxcfs/* &&
rm -rf ~/jail-pmxcfs/

[1] https://forum.proxmox.com/threads/154569/
[2] https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)
[3] https://manpages.debian.org/bookworm/coreutils/chroot.8.en.html

 

[VGusev2007]

You're best of the best. Thank a lot for your efforts. I'll use it also in my other project.

 

[xavi.verdnatura]

What do you think about this one-liner @esi_y:

proxmox-backup-client backup etc-pve.pxar:/etc/pve --repository $IP_MACHINE:$POOL

 

[waltar]

What do you think about this one-liner @esi_y:

esi_y isn't there anymore but you can reach him at reddit.com in some forums there as esiy0676.

 

[kobuki]

For whomever is interested and finds this thread via search, I've made a simple python script (parts courtesy of ChatGPT) that is able to extract the complete file tree from a config.db. The script can be used without Proxmox, Python 3 and a few basic Python modules are the requirements. See here.