Is there a way to backup the PVE Host to the Proxmox Backup Server (PBS)?
- Thread starter logui
- Start date
Backing up the host is not super difficult, but depends a bit on some of the details of how your system is installed. For a host that uses ZFS as root, I do something along these lines:
You can obviously make all sorts of customizations, depending on your local needs. And if you don't use ZFS, then you need to figure out on your own, how to create atomic snapshots. If you have other ZFS filesystems that also need to be backed up, then don't forget to include them here.
Recovery is the big elephant in the room. That's potentially more difficult. If you just want to recover some part of the file hierarchy, that's easy enough to do. But if you want to bring up an entirely new machine, you have to decide what to do about boot strapping, and what to do about differences in hardware.
You could probably boot a Debian rescue disk, manually partition your drives and/or create appropriate ZFS pools and filesystems. Then use the proxmox-backup-client to restore the files. Afterwards, chroot into the new filesystem and configure your boot loader. You also might have to edit /etc/network/interfaces and possibly adjust /etc/systemd/network/*.link files, if you assign stable network interface names based on MAC address.
Editing /etc/hostname wouldn't be a bad idea either.
This is all a little tricky and depending on how good you are with Linux, it could take you a couple of hours to fully restore a system after a catastrophic failure. If you have access to spare hardware, I would try to practice this, or I would attempt to make a custom-built rescue image that you can boot into in order to automate this process. But the specifics will depend a lot on your local needs.
Bash:
#!/bin/bash
# Configure where to store the backup and what to exclude, as it changes
# regularly and only contains emphemeral data.
pbspasswd="/etc/pbs-passwd"
pbshost="pbs.lan"
pbsns="pve-host"
pbsvol="pve-sys"
exclude=( '/dev/shm' '/tmp' '/run' '/var/cache' '/var/lib/rrdcached'
'/var/tmp' '/var/lib/vz#' '/var/log#' )
# Convert the list of exclusions into command line arguments for the PBS
# client. Treat directories like /var/log special, as we want to keep some
# of the directory structure and permissions in the backup, but want to
# discard all of the actual data.
exdirs=
for ex in "${exclude[@]}"; do
exdirs="${exdirs} --exclude ${ex%#}/?*"
[ "${ex%#}" != "${ex}" ] &&
exdirs="${exdirs} --exclude ${ex%#}/**/?* --exclude !${ex%#}/**/"
done
# Take advantage of ZFS to create an atomic snapshot for backing up.
# We keep this snapshot around afterwards, as it is useful for quickly
# repairing accidentally damaged systems.
root="$(zfs list / | awk 'NR==2{ print $1 }')"
zfs destroy "${root}@backup" >&/dev/null || :
zfs snapshot "${root}@backup"
[ -d "/.zfs/snapshot/" ] || zfs set snapdir=visible "${root}"
# The credentials for accessing the PBS server should be stored in
# /etc/pbs-passwd
export PBS_REPOSITORY="$(sed -n 2p <"${pbspasswd}")@${pbshost}:${pbsvol}"
export PBS_PASSWORD_FILE="${pbspasswd}"
# Create a new namespace if it doesn't exist yet, then backup our snapshot
# to the PBS server. Exclude ephemeral data, as it just fills up the server
# and likely won't deduplicate well.
proxmox-backup-client namespace list 2>/dev/null | egrep "^${pbsns}$" >/dev/null ||
proxmox-backup-client namespace create "${pbsns}"
(set -f
proxmox-backup-client backup "proxmox-root.pxar:/.zfs/snapshot/backup" --ns "${pbsns}" \
--change-detection-mode=metadata ${exdirs})You can obviously make all sorts of customizations, depending on your local needs. And if you don't use ZFS, then you need to figure out on your own, how to create atomic snapshots. If you have other ZFS filesystems that also need to be backed up, then don't forget to include them here.
Recovery is the big elephant in the room. That's potentially more difficult. If you just want to recover some part of the file hierarchy, that's easy enough to do. But if you want to bring up an entirely new machine, you have to decide what to do about boot strapping, and what to do about differences in hardware.
You could probably boot a Debian rescue disk, manually partition your drives and/or create appropriate ZFS pools and filesystems. Then use the proxmox-backup-client to restore the files. Afterwards, chroot into the new filesystem and configure your boot loader. You also might have to edit /etc/network/interfaces and possibly adjust /etc/systemd/network/*.link files, if you assign stable network interface names based on MAC address.
Editing /etc/hostname wouldn't be a bad idea either.
This is all a little tricky and depending on how good you are with Linux, it could take you a couple of hours to fully restore a system after a catastrophic failure. If you have access to spare hardware, I would try to practice this, or I would attempt to make a custom-built rescue image that you can boot into in order to automate this process. But the specifics will depend a lot on your local needs.
Here's the script that I am currently using, it focusses on backing up the host configuration, not the payload (VMs, CT, etc.).
The file is located at
Content of that file:
Also, since I use encrypted backups, the encryption key (file) needs to be saved as
Bash:
#!/bin/bash
if [ -f /etc/pve/local/pve-backup.env ] ; then
source /etc/pve/local/pve-backup.env
else
echo "File /etc/pve/local/pve-backup.env missing" > /dev/stderr
exit 1
fi
/usr/bin/proxmox-backup-client backup root.pxar:/ \
--crypt-mode encrypt \
--keyfile /etc/pve/pve-backup.json \
--exclude /bin \
--exclude /boot \
--exclude /dev \
--exclude /lib \
--exclude /lib64 \
--exclude /local-zfs \
--exclude /lost+found \
--exclude /mnt \
--exclude /opt \
--exclude /proc \
--exclude /run \
--exclude /sbin \
--exclude /sys \
--exclude /tmp \
--exclude /usr \
--exclude /var/lib/lxcfs \
--include-dev /etc/pve \
--backup-type host \
--skip-lost-and-foundThe file is located at
/usr/local/sbin/pve-backup.sh and executed by cron daily. The referenced file /etc/pve/local/pve-backup.env is a simple key/value file, containing the Proxmox node specifics. That way I can use the same script on all my PVE nodes unchanged, while only having to adjust the host specific configuration in /etc/pve/local/pve-backup.env .Content of that file:
Bash:
export PBS_REPOSITORY=<API TOKEN USER>!<API TOKEN NAME>@<PBS HOST>:<DATASTORE>
export PBS_PASSWORD=<API TOKEN>
export PBS_FINGERPRINT=<PBS HOST FINGERPRINT>Also, since I use encrypted backups, the encryption key (file) needs to be saved as
/etc/pve/pve-backup.json. If you don't use encryption, simply remove the lines "--crypt-mode" and "--keyfile".
Last edited:
Thanks, backup works just fine. But what is the method to restore such a backup ?
Restoring is more complex, depends a lot more on the specifics of your situation, and that's most likely the reason why Proxmox doesn't officially have any support for backing up the host. It's not that it's difficult per se. But it's difficult to come up with a solution that works for everyone each and every time.
In my case, I've used it a few times to restore a host into a running Chromebook (yes, I know I'm strange; don't ask why). That's probably the hardest target to restore, as you can't even boot from rescue media. So, I had to jump through extra hoops.
In the end, I started out with the default Debian container that ChromeOS installed for me (in hindsight, a minimal Alpine container would probably be even better). I then manually installed and configured the Proxmox client software, FUSE support, and rsync. This allowed me to mount the backup as a file system. And I could then use rsync to copy it over my running system. When done, I rebooted, and executed rsync one more time, just in case anything didn't copy properly on the first run.
This worked surprisingly well. Much better than I had expected. The only thing that required manual repair were permissions on things like ping. I don't know whether they're stored incorrectly in PBS or if rsync dropped capabilities. But that's only a very small number of files and easy to fix.
If you are restoring to raw metal, things are presumably easier. Either boot from rescue media or into the initramfs in rescue mode.
If this is something you expect to do regularly, it wouldn't be difficult to script. But specifics depend a lot on your local requirements. I could see a PXE-based solution that restores from the network.
One of the things you'll have to take care of is the creation of disk partitions for your filesystem(s), and making sure that the ids for these partitions or drives are referenced in all the important places. That might mean the ZFS pool, or things like /etc/fstab and /etc/kernel/cmdline. Don't forget to run proxmox-boot-tool
In my case, I've used it a few times to restore a host into a running Chromebook (yes, I know I'm strange; don't ask why). That's probably the hardest target to restore, as you can't even boot from rescue media. So, I had to jump through extra hoops.
In the end, I started out with the default Debian container that ChromeOS installed for me (in hindsight, a minimal Alpine container would probably be even better). I then manually installed and configured the Proxmox client software, FUSE support, and rsync. This allowed me to mount the backup as a file system. And I could then use rsync to copy it over my running system. When done, I rebooted, and executed rsync one more time, just in case anything didn't copy properly on the first run.
This worked surprisingly well. Much better than I had expected. The only thing that required manual repair were permissions on things like ping. I don't know whether they're stored incorrectly in PBS or if rsync dropped capabilities. But that's only a very small number of files and easy to fix.
If you are restoring to raw metal, things are presumably easier. Either boot from rescue media or into the initramfs in rescue mode.
If this is something you expect to do regularly, it wouldn't be difficult to script. But specifics depend a lot on your local requirements. I could see a PXE-based solution that restores from the network.
One of the things you'll have to take care of is the creation of disk partitions for your filesystem(s), and making sure that the ids for these partitions or drives are referenced in all the important places. That might mean the ZFS pool, or things like /etc/fstab and /etc/kernel/cmdline. Don't forget to run proxmox-boot-tool
Last edited:
Code:
#!/bin/bash
if [ -f ~/backup.env ] ; then
source ~/backup.env
else
echo "File ~/backup.env missing" > /dev/stderr
exit 1
fi
/usr/bin/proxmox-backup-client backup root.pxar:/ \
--exclude /bin \
--exclude /boot \
--exclude /dev \
--exclude /lib \
--exclude /lib64 \
--exclude /local-zfs \
--exclude /lost+found \
--exclude /mnt \
--exclude /opt \
--exclude /proc \
--exclude /run \
--exclude /sbin \
--exclude /sys \
--exclude /tmp \
--exclude /zstor \
--exclude /usr \
--exclude /var/lib/lxcfs \
--include-dev /etc/pve \
--backup-type host \
--skip-lost-and-foundbackup.sh and backup.env located@ /root
backup.env is user, password and fingerprint copied from the <datastore> <your backup> <show connection info> in pbe
setup as a cron job if you dare
Last edited: