Proxmox backups with restic: ResticBackup.sh and ResticRestore.sh
Two bash scripts, one shared config.cfg, one restic repository on an external disk. Deduplicated, encrypted, restorable without detours - and single files reachable without unpacking hundreds of gigabytes first.Requirement: existing snapshots
The backup does not create ZFS snapshots of its own. For ZFS volumes - VM disks as well as LXC rootfs - it always uses the newest existing snapshot of the dataset, both for zfs send and as the read source for RAW. If there is none, the disk is skipped instead of writing a smeared image (RAW_REQUIRE_SNAPSHOT). If it is older than 24 hours you get awarning, and that state is what ends up in the backup.So how current a backup is depends on the snapshot mechanism on the host: a cron job running cv4pve-autosnap or Proxmox's own snapshots - as long as one is taken shortly before the backup run.
Exceptions: on LVM and LVM-thin the script creates a temporary snapshot itself and removes it afterwards; on directory storage it reads live, without a consistency guarantee.
Backup medium
- the target is found by UUID, not by a fixed path; several UUIDs = rotating media
- a log records which disk ran and warns when the same one was used for 30 runs straight
- after the run: fstrim, standby (hdparm -y), unmount
Formats
- RAW (default): the volume block by block as an image, taken from a ZFS or a temporary LVM snapshot. Every backup stands on its own - no chain, restore in one step
- zfs send (optional): per disk (ZFS_SEND_DISKS="100:scsi1") or for everything. Full plus differentials; a new full is forced when the last one is too old, its base is gone, or the diff grows past a set share of the full. Saves read I/O on large data disks but needs room for two complete generations - the run itself points out worthwhile candidates
- File based: LXC rootfs via a read-only snapshot, LOCAL_DIRS, machine configs
- a config-id tag brackets everything belonging to one run - a restore takes exactly those disks, not the ones closest in time
Space
- the requirement is calculated exactly, not estimated: a backup not yet in the repo needs its full source size, an existing one a percentage on top; for RAW the real allocation counts, not the logical disk size
- if that is not enough, whole backup days are released, oldest first: a full takes its diffs, their configs and the always-full volumes with it; the newest backup of a group and anything from ZFS_SEND_DISKS stay
- retention via KEEP_DAILY/WEEKLY/MONTHLY plus prune
Integrity and warnings
- restic never re-reads a chunk it has already stored, so a rotted pack would go unnoticed. Hence the rotating read-back: after RESTIC_CHECK_ROTATE_RUNS runs the whole repository has been read once, with the counter living on the medium
- warnings that never abort: full host filesystems, long uptime, little space left on the target
- an overview of all backups including restore size at the end of the run, also in the mail
Restore
- --list with filters (machine, type, name, run), optionally with size; --list-disks <VMID> for the state of one machine
- a restore needs an explicit --target on a free ID - existing machines are never overwritten, deliberately without an override
- --machine = latest run, --date = latest before a cut-off, --snap = exactly one; target volumes via pvesm alloc, differentials pull in their full automatically
Single files
- --mount <SNAP-ID>: repository via FUSE, a RAW image as a read-only loop device with its partitions mounted (LVM inside is activated), LXC/LOCAL straight as a file tree. No extra disk space needed
- --attach-raw <SNAP-ID> --vm <VMID>: for everything the host cannot read (Windows, Storage Spaces, LUKS with the key inside the guest) - the image goes into the running VM as a SCSI disk
- --rw puts a qcow2 overlay in front, served to the VM as /dev/nbdN by qemu-nbd; writes land there and are discarded, the backup stays untouched
- the same VM works too: Windows keeps the copy offline because of the duplicate signature - right-click the disk, "Online", and the new signature goes into the overlay
- Ctrl+C releases everything; after a hard kill --detach-raw catches up (VM config, loop/nbd device, FUSE mount, overlay, temp dir) - and only touches what the scripts created themselves
Installation and Configuration
Dependencies:
Code:
apt install restic jq pvCopy all three files to /usr/local/bin/ and drop the .txt extension.
Code:
chmod +x ResticBackup.sh
chmod +x ResticRestore.shFind the UUID of the external disk and put it into /usr/local/bin/config.cfg.
Code:
blkidEvery commented-out parameter in config.cfg is shown with the default it uses when left
alone - the file runs as it is.
Back up all VMs, all LXC containers and the optional LOCAL_DIRS:
Code:
./ResticBackup.shBack up machine 115 only (--dry-run and -v work as well):
Code:
./ResticBackup.sh 115Restore:
Code:
./ResticRestore.sh --help
# e.g.
./ResticRestore.sh --machine 115 --target 9115Runtime:
Code:
VM:
600 GB machine storage
300 GB used
5 GB diff per day
Backup-Storage: 930 GB
| Read I/O VM | Write I/O Backup | max Backup count | runtime
| 500 MB/s | 100 MB/s | on disk |
------------------------------------------------------------------------------------
VZDump | 600 GB | 250 GB * | | 42 min
------------------------------------------------------------------------------------
20-backup-sum | 12 TB | 5 TB | 2 ** | 840 min
RAW first time | 600 GB | 250 GB * | | 42 min
RAW following | 600 GB | 5 GB | | 20 min
------------------------------------------------------------------------------------
20-backup-sum | 12 TB | 345 GB | > 100 *** | 422 min
zfs_s 1. full | 300 GB | 300 GB * | | 50 min
zfs_s 1. diff | 5 GB | 5 GB | | 1 min
zfs_s 2. diff | 10 GB | 10 GB | | 2 min
zfs_s 3. diff | 15 GB | 15 GB | | 2.5 min
...
zfs_s 9. diff | 45 GB | 45 GB | | 7.5 min
zfs_s 2. full | 300 GB | 300 GB | | 50 min
zfs_s 10. diff | 5 GB | 5 GB | | 1 min
...
------------------------------------------------------------------------------------
20-backup-sum | 1.1 TB | 1.1 TB | 13 **** | 175 min
* 300 GB used storage can compress to approx 250 GB; if you store via zfs_send, further reduction is not possible
** I you use the prune process of VZdump, which only prune after backup your total number is 2.
You can increment this to 3 if you use your own prune process, which prune directly before creation
*** (930 - 250)/5 = 137, but you would not conserve 137 dailys,
so the difference between individual backups eventually become larger than 5 GB
****Initial you can add 17 backups, but worst case would be: 300+40+45+300+5+10+15+20+25+30+35+40+45 = 910 => 13 backups
Last edited: