[SOLVED] HOW TO BACKUP PROXMOX CONFIGURATION FILES.

[noobs]

Okay - but it isn't simply cp -r /mnt/pve/ /etc/ - that threw a bunch of errors.
It seems that the hostname being identical is of critical importance, but doesn't get mentioned.

I had to make sure I only copied the conf files from the backed up pve directory and sub directories, and then import the pools, and then if the hostname has changed you need to specify the host the pools belong to again, then there were some network shenanigans which required the guest adapters be removed and re-added.

I'll try to get a lab going again and break things there rather than production.

 

[adresner]

While I see there are ways to backup your host, i'm on the same page with others that it should be top priority for Proxmox to have a backup host option within the Backup under Datacenter. Backups, Backups, Backups.

All of us should be able to easily, confidently, backup our hosts and restore them with a solution from Proxmox, not install git, etc.

Proxmox? Backup our hosts solution please.

 

[keeka]

Some of these solutions seem over complex to me but I may be missing something.
My PVE host backup strategy has been to create simple backup archives that are easy to use should I need to selectively restore config:

rsnapshot /etc/ (one_fs=0), /root, /usr/local (where I try to make a point to store all custom scripts etc).
`dpkg --get-selections >packages`

Then, should I need to restore either partial or complete, it is a simple case of
rsyncing from backup, those parts I need, to current or a clean installation.
Install any non-default packages.
Adjust storage config if need be (e.g. new hardware).
Adjust network config if needed.
Restore CT/VM from PVE guest backups.

What I am not sure about is locations such as /var/lib/lxc & /var/lib/pve-cluster.
AIUI /etc/pve is the definitive config and the locations such as the above are only relevant to running state of the host/guests. Not needed in a backup. Is that assumption wrong?

 

[Dunuin]

AIUI /etc/pve is the definitive config and the locations such as the above are only relevant to running state of the host/guests. Not needed in a backup. Is that assumption wrong?

Stuff like aliases, IP sets, security groups used by guests actually aren't part of the VM/LXC backups but part of "/etc/pve/firewall/datacenter.cfg".
And other important files like encryption keys (for example when using PBS with encryption), SMB share passwords are also stored in the "/etc/pve/" folder. And datacenter/node firewall rules, ... "/etc/pve" is actually the most important folder as most of the PVE config files are stored there.

 

[keeka]

Stuff like aliases, IP sets, security groups used by guests actually aren't part of the VM/LXC backups but part of "/etc/pve/firewall/datacenter.cfg".
And other important files like encryption keys (for example when using PBS with encryption), SMB share passwords are also stored in the "/etc/pve/" folder. And datacenter/node firewall rules, ... "/etc/pve" is actually the most important folder as most of the PVE config files are stored there.

I understand that /etc/pve is critical. However mention has been made of /var/lib/pve-cluster. Whilst it appears to store config data, I was under the impression it was part of the running state of PVE and not critical to backup. Is that correct? I have not yet used my backup to restore on top of a new install. I have imaged the PVE root partition too. EDIT: Rather I imaged the entire PVE system disk. It's only a few GB since it holds no storage.

 

[keeka]

While I see there are ways to backup your host, i'm on the same page with others that it should be top priority for Proxmox to have a backup host option within the Backup under Datacenter. Backups, Backups, Backups.

All of us should be able to easily, confidently, backup our hosts and restore them with a solution from Proxmox, not install git, etc.

Proxmox? Backup our hosts solution please.

Due to the versatility of setup, I'm wondering if such a one size fits all system backup/restore utility for PVE is feasible.

 

[Dunuin]

I understand that /etc/pve is critical. However mention has been made of /var/lib/pve-cluster. Whilst it appears to store config data, I was under the impression it was part of the running state of PVE and not critical to backup. Is that correct? I have not yet used my backup to restore on top of a new install. I have imaged the PVE root partition too. EDIT: Rather I imaged the entire PVE system disk. It's only a few GB since it holds no storage.

/etc/pve is a special folder. All of the PVE configs are actually stored in a SQLite DB called /var/lib/pve-cluster/config.db. Then there is the pmxcfs filesystem which uses fuse to map those configs stored in the config DB as config files in the mountpoint "/etc/pve".
So you can either backup the config.db or the "/etc/pve" folder to backup your PVE config files.

 

[adresner]

I understand your point, not saying your wrong since there are so many ways to customize PVE.. however there should be a backup like PFSense has, where we can choose certain areas and maybe even directories to backup and simply restore. A download, something one can easily upload upon re-install. My 2 cents, this should be a priority as a way of providing a "finished" product. At my office, I use Hyper-V and Veeam, I wish Veeam supported Proxmox? i know there is a way, but thats the point, i want something fully supported when it comes to a backup.

After writing that post this morning, I dug up a 1LTR that has Hyper-V on it. installed Proxmox backup, say it backs up "Proxmox Backup Server is an enterprise backup solution, for backing up and restoring VMs, containers, and physical hosts." However, I do not see any option to backup the host, I'll check again after I get the kids to sleep.

I am really enjoying Proxmox, learning a lot! bough a subscription to support the company, this lack of a way to backup the host configuration and settings is disappointing but also points to the complexity of a complete backup solution, however like I mentioned earlier, its doable like PFSense where you tick off the areas you want to backup and option to choose certain folders.

Hope the powers at Proxmox consider this soon

edit: Jealous of the guys who can use rsync and the other tools that people are mentioning here to backup this directory and that up. For a lot of us, thats gonna be something to figure out but when we go thru recovery, it's not something we use all the time. We really need a proper backup.

 

[Dunuin]

I understand your point, not saying your wrong since there are so many ways to customize PVE.. however there should be a backup like PFSense has, where we can choose certain areas and maybe even directories to backup and simply restore. A download, something one can easily upload upon re-install.

Yeah, but the problem here is that PVE is not an appliance like TrueNAS or pfsense where it is easy to export and import a config file that contains all the configs. You are not supposed to do anything with pfsense that is not part of the webUI/CLI so you always exactly know what needs to be backed up and every installation is identical, except for some standardized config files.
But PVE is a whole OS where you are allowed to do stuff like installing steam on it bare metal to play some games, run an apache bare metal to host some websites or whatever. Not that it would be recommended to do it that way, but you can easily do this and it is fine if you want to do that. Everyone can run a totally custom PVE installation (mine is for example full system encrypted which PVE got no support for but still works fine) so it's not easy to backup and restore a PVE installation when not backing up the whole system disk with all of its content.

 

[keeka]

/etc/pve is a special folder. All of the PVE configs are actually stored in a SQLite DB called /var/lib/pve-cluster/config.db. Then there is the pmxcfs filesystem which uses fuse to map those configs stored in the config DB as config files in the mountpoint "/etc/pve".
So you can either backup the config.db or the "/etc/pve" folder to backup your PVE config files.

Thanks. for that info. I was aware that /etc/pve was a fuse mount backed by the proxmox userland application. I didn't know that that sqlite file was part of it.
So, if one were to restore config to a non-running PVE system, they must restore that sqlite db. I have added it to my rsnapshot config!

 

[Dunuin]

So, if one were to restore config to a non-running PVE system, they must restore that sqlite db. I have added it to my rsnapshot config!

Jup as the /etc/pve then would be empty without the pvecluster service running. See here: https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)#_recovery

 

[adresner]

Yeah, but the problem here is that PVE is not an appliance like TrueNAS or pfsense where it is easy to export and import a config file that contains all the configs. You are not supposed to do anything with pfsense that is not part of the webUI/CLI so you always exactly know what needs to be backed up and every installation is identical, except for some standardized config files.
But PVE is a whole OS where you are allowed to do stuff like installing steam on it bare metal to play some games, run an apache bare metal to host some websites or whatever. Not that it would be recommended to do it that way, but you can easily do this and it is fine if you want to do that. Everyone can run a totally custom PVE installation (mine is for example full system encrypted which PVE got no support for but still works fine) so it's not easy to backup and restore a PVE installation when not backing up the whole system disk with all of its content.

And mine isn't custom, I can probably rebuilt it faster than restoring a backup...and thats what my plan is right now. Make notes, be ready if I need to rebuild... and I understand we aren't downloading an XML, it's gonna be a backup of files...just conceptually.. choose which areas you want to backup and for that, i would argue there should be a backup... even dare I say an image of ones install you can easily restore.. that covers your entire install.

 

[Dunuin]

One thing I don't understand is, why there is no webUI integration to back up some folders. As you already can use the proxmox-backup-client in CLI to do host backups and backup any file, folder or block device to your PBS server. I for example already use that to backup my PVE config files as well as the whole system disk on block level. It's just not simple to set that all up so most people won't use it.

 

[LnxBil]

If you have a cluster, you'll already have a distributed configuration copy, which is also nice. Normally not all cluster nodes fail simultaneously.

 

[keeka]

One thing I don't understand is, why there is no webUI integration to back up some folders. As you already can use the proxmox-backup-client in CLI to do host backups and backup any file, folder or block device to your PBS server. I for example already use that to backup my PVE config files as well as the whole system disk on block level. It's just not simple to set that all up so most people won't use it.

Do you perform the root fs block level backups on running system? Also how do you go about restoring a PVE root fs from block level backup stored on PBS?

 

[Dunuin]

Do you perform the root fs block level backups on running system? Also how do you go about restoring a PVE root fs from block level backup stored on PBS?

No, I only backup the config files while PVE is running. For backing up the system disk I installed Debian 11 to a USB pen drive I keep always plugged in.
To then backup the system disks on block level to my PBS I reboot the server, boot into that Debian and run a script that will use the proxmox-backup-client to backup those two disks of my rpool ZFS mirror.

So some kind of official "Proxmox Backup & Recovery ISO" with a GUI and/or CLI UI would be great that would allow us to backup any host to a PBS and allow us to restore it later. You can do that all on your own but would be great if there would be something ready to use and well tested directly from the Proxmox team.

 

[keeka]

No, I only backup the config files while PVE is running. For backing up the system disk I installed Debian 11 to a USB pen drive I keep always plugged in.

Thanks @Dunuin. With the pve root fs being on a logical volume, I wonder if it's possible to do an lvm snapshot and make that the source of your block level backup to PBS?

 

[Dunuin]

Thanks @Dunuin. With the pve root fs being on a logical volume, I wonder if it's possible to do an lvm snapshot and make that the source of your block level backup to PBS?

Should be possible. But that then only would include that root LV or ZFS root dataset. Not other stuff like the two bootloader partitions and so on. So you won`t get an image that you could write to an empty disk to restore your PVE installation. Would then be more complex to do a restore.

Some people here already use snapshots to be able to rollback the root filesystem.

 

[LnxBil]

Thanks. for that info. I was aware that /etc/pve was a fuse mount backed by the proxmox userland application. I didn't know that that sqlite file was part of it.
So, if one were to restore config to a non-running PVE system, they must restore that sqlite db. I have added it to my rsnapshot config!

Haven't you read through the whole thread? It was already mentioned in comment 33 along with other good advice.

 

[keeka]

Haven't you read through the whole thread? It was already mentioned in comment 33 along with other good advice.

Oh yes. However the significance of that file slipped my notice because I was unaware of how it was involved in presenting the contents of /etc/pve.

root@odin:~# systemctl stop pve-cluster.service
root@odin:~# ls -l /etc/pve/
total 0

FWIW my backup strategy now, since I do not run a cluster, is:
Block level backup of the system disk at least every significant upgrade.
Periodic filesystem backup of key files (since I now have a better understanding of where and how the config is stored/generated).
Continue to avoid installing any extraneous software on the hypervisor.