Snapshot proxmox installation itself

[ieronymous]

Hi

Is there a way to snapshot or backup the installation in order to have a way to revert back if something happens from updates?
The installation is on a mirror zfs in uefi mode in case that matters.

Thank you in advance

PS By backup I dont mean to take an image of the disk containing the installation itself.

 

[dylanw]

Hi,

You should be able to do this using ZFS snapshots, but be careful not to accidentally revert anything you want to keep, such as VMs or configuration files. It might be a better idea to simply backup /etc and particullarly /etc/pve, so that you retain configuration settings in case you need to reinstall the system.
If after an upgrade, a newer kernel causes any problems, you can instead boot into an older kernel to make the system behave normally again. I believe that Proxmox VE retains the 3 most recently installed kernels by default.

 

[Dunuin]

If you want a backup of your complete system disk as an image (inculding bootloader and all that) you could boot into a clonezilla USB stick and use that to save that image to a NAS or USB disk.
And if you are also running a PBS server you can create a debian USB stick and use the proxmox-backup-client to create an image of you system disk and store it in your PBS datastore.
I use the second option which works really nice, because of the PBS deduplication that drive only needs to be saved once. So the mirrored system disk consumes no additional space and additional later backups neighter.

 

[ieronymous]

Thank you for your answer

You should be able to do this using ZFS snapshots,

Like how? where that image would be stored?

but be careful not to accidentally revert anything you want to keep, such as VMs or configuration files.

.... but the point is to revert back. After update I ll check everything as long as the update finishes. So if something goes way wrong I would like to revert back everything.

It might be a better idea to simply backup /etc and particullarly /etc/pve, so that you retain configuration settings in case you need to reinstall the system.

... that I know and don t like as a way of doing things

If you want a backup of your complete system disk as an image (inculding bootloader and all that) you could boot into a clonezilla USB stick and use that to save that image to a NAS or USB disk.

This is what I was doing in a non production level where the installation was only one ssd and I was booting to rescuezilla (I prefer it ) and restore from there. Now with a dual disk the procedure takes longer I need a larger disks to have images of both disks and there is the possibility to restore the image (even the same) to the wrong disk with a different uuid and then you need to do that all over again. Too much trouble for an update (even though I am now at 102 pending updates and starting to get nervous)

And if you are also running a PBS server you can create a debian USB stick and use the proxmox-backup-client to create an image of you system disk and store it in your PBS datastore.

I would be interested in that but I can t understand what you are saying and how is this done, At home I have a pbs but still dont know the steps of what you are describing.

PS1 Please don t point out to documentation again. I have been reading them for 2+ years have my notes on text files on many things except that. Backing up proxmox itself.

PS2 There was a video of a German channel (something like zfs rock) and was describing a script that snaps the installation every 15 mins \ every day \ once per week \ once per month (via a cron job) but I never had an answer about how I could to that manually (which were the commands, paths of files...etc).

 

[Dunuin]

Like how? where that image would be stored?

ZFS stores no image files. ZFS snapshots are part of the disk management itself and will not serve as a backup (or only if you use ZFS replication to replicate your snapshots from one pool to another ZFS pool). And only partition 3 of the system disk would be ZFS, so the bootloader and ESP of the first 2 partitions can't be restored with that. But ZFS snapshots could be useful if an update failed and you just want to do a fast rollback.

I would be interested in that but I can t understand what you are saying and how is this done, At home I have a pbs but still dont know the steps of what you are describing.

PS1 Please don t point out to documentation again. I have been reading them for 2+ years have my notes on text files on many things except that. Backing up proxmox itself.

PS2 There was a video of a German channel (something like zfs rock) and was describing a script that snaps the installation every 15 mins \ every day \ once per week \ once per month (via a cron job) but I never had an answer about how I could to that manually (which were the commands, paths of files...etc).

You do a normal linux installation to a USB stick (I used Debian 11 but any linux should work) and set it up (network and so on). Then you install the proxmox-backup-client package from the client-only PBS repo. The PBS is CLI only, so you need to write yourself a script to do the backup for you.

Mine for example looks like this:

#!/bin/bash

crypt="none"
bcktype="host"
bckid="WhatYourHostShouldBeNamed"

declare -a blkdev_id
declare -a blkdev_path
# first system disk (sda)
blkdev_id[0]="firstDrive"
blkdev_path[0]="/dev/disk/by-id/ata-your_1st_disk"
# second system disk (sdb)
blkdev_id[1]="secondDrive"
blkdev_path[1]="/dev/disk/by-id/ata-your_2nd_disk"

# repo configs
repo_user="YourPbsUser"
repo_usertype="pbs"
repo_token="YourTokenName"
repo_host="Your.PBS.IP.ADDR"
repo_port="8007"
repo_datastore="yourDatastoreName"

PBS_PASSWORD="yourPbsTokenKey"
export PBS_PASSWORD
#PBS_ENCRYPTION_PASSWORD="yourPass"
#export PBS_ENCRYPTION_PASSWORD
PBS_FINGERPRINT="yourPbsFingerprint"
export PBS_FINGERPRINT

proxmox-backup-client backup "${blkdev_id[0]}.img:${blkdev_path[0]}" "${blkdev_id[1]}.img:${blkdev_path[1]}" --repository "${repo_user}@${repo_usertype}!${repo_token}@${repo_host}:${repo_port}:${repo_datastore}" --backup-type "${bcktype}" --crypt-mode "${crypt}"

# you can find out the UUIDs using this:
# ls -l /dev/disk/by-id

# To write a backuped image from PBS to a disk the command should look like this:
# proxmox-backup-client restore host/pve/YYYY-MM-DDTHH:MM:SSZ firstDrive.img -  --repository YOUR.PBS.IP.ADDR:YourStorageNAme | dd of=/dev/disk/by-id/ata-your_1st_disk bs=4096

Wanted to write a real script with UI for creating profiles, selection of drives and snapshots to restore from/to, doing backups but that looked like alot of work so I didn't done that yet. Writing an disk to image or image to disk is basically a one liner. So you just need to read the PBS manual to understand the backup-client usage and choose the correct arguments for that oneliner.

Best you setup a dedicated user for your backups and create a Token for that user and only allow the token to backup and restore. Then you can use that token for authentification.

 

[ieronymous]

ZFS stores no image files. ZFS snapshots are part of the disk management itself and will not serve as a backup

Yeah you re right . I just didn t express correctly that is why the misunderstanding between backup - snaphot .

so the bootloader and ESP of the first 2 partitions can't be restored with that.

Makes sense since snapshot isnt doing images but does create some kind of file in the vzdump path,

But ZFS snapshots could be useful if an update failed and you just want to do a fast rollback.

Yes that is why I am trying to achieve that, Are you talking about the pbs way or another one via cli direct?

A about the pbs explanation I didnt expect to be so thorough . Very nice thanks but that is above my league for now. I aint there yet with scripting.

 

[ieronymous]

In case someone find this useful ... for me almost worked but not the way I need to ....

Backup proxmox using UrBackup

Pre-requirements
A working Proxmox VE host
A spare server for running as UrBackup server with big enough storage
The Proxmox VE host and the UrBackup Server is connected and in the same LAN
1 Install UrBackup Server
The server can be Windows or Linux (Debian/Ubuntu etc.)

1.1 For Windows Server
1.1.1 Download the exe or msi file from here: UrBackup Windows Server [1]

1.1.2 Execute the downloaded binary file, install the UrBackup server

1.1.3 Open any browser from the server, navigate to “localhst:55414” or “127.0.0.1:55414”

Note 1: The web interface is available at port 55414. To restrict the access you have to create
an admin account in Settings->Users. Without this account everyone can access all backups using the web interface.

Note 2: This web interface is accessible from other device within the same LAN

1.2 For Linux (Ubuntu) Server
1.2.1 You can simply use following commands to install it

sudo add-apt-repository ppa:uroni/urbackup
sudo apt update
sudo apt install urbackup-server
or

wget https://hndl.urbackup.org/Server/2.4.12/urbackup-server_2.4.12_amd64.deb
sudo dpkg -i urbackup-server_2.4.12_amd64.deb
sudo apt install -f
Note 1: It has some dependencies which you can automatically resolve by running apt-get -f install.
If it does not work you probably chose the wrong packet from stable/testing/unstable. [2]

Note 2: The web interface is available at port 55414. To restrict the access you have to create an admin account
in Settings->Users. Without this account everyone can access all backups using the web interface. [2]

Note 3: This web interface is accessible from other device within the same LAN

2 Install UrBackup Client on Proxmox VE host
2.1 Login to Proxmox VE terminal directly or via SSH or via web gui -> Shell

2.2 Execute following command to install UrBackup client

TF=$(mktemp) && wget "https://hndl.urbackup.org/Client/2.4.10/UrBackup Client Linux 2.4.10.sh" -O $TF && sh $TF; rm -f $TF
Note: Unless you are sure you want to use snapshot, dattobd require manual installation, LVM requires free space that
equals to the size you are backing up. Choose not to use snapshot here if you are not sure even it says you can
use dattobd and LVM.

3 Configure the UrBackup Server (and Proxmox host)
3.1 Before you start to configure UrBackup server, you need to make sure Proxmox VE host’s firewall is disabled,
or you will have to open ports for UrBackup server, where are listed in following tables

The Server binds to following default ports:

Port Usage Incoming/Outgoing Protocol
55413 FastCGI for web interface Incoming TCP
55414 HTTP web interface Incoming TCP
55415 Internet clients Incoming TCP
35623 UDP broadcasts for discovery Outgoing UDP
[3]
The Client binds to following default ports (all incoming):

Port Usage Protocol
35621 Sending files during file backups (file server) TCP
35622 UDP broadcasts for discovery UDP
35623 Commands and image backups TCP

[3]
3.2 Login to UrBackup web interface

3.3 Check if the Proxmox VE host appear under “Status” tab, if not, wait 30 – 60 minutes, if you can see the
Proxmox VE is displayed under status tab, proceed to next step

3.4 Click on “Settings”

3.5 Click on “Client Settings”, then click on the name of Proxmox VE host

3.6 Check “Separate settings for this client”, make necessary changes for “File Backups”, no need to worry about
“Image Backups” since it is not supported on Linux. (Do not forget to click on “Save” button)

Important: Make sure for File Backups, you exclude zfs pool if you are backing up the root “/”

e.g. Backing up the root file system “/”

Excluded files (with wildcards): /rpool/*;/mnt/proxmox-hostname/*;/tmp/*
Default directories to backup: /

Note: You probably want to exclude “/mnt/data/*” as well.

If you are not doing any modification to system files, you can just backup following folder

/etc
What’s inside

# Note: replace <proxmox-hostname> with your node name

# Virtual machine .conf, LXC .conf, SSL certificates, storage related configuration, system settings etc.
/etc/pve/nodes/<proxmox-hostname>/

# Virtual machine .conf files
/etc/pve/nodes/<proxmox-host-name>/qemu-server/

# LXC .conf files
/etc/pve/nodes/<proxmox-hostname>/lxc/

# Mounting information
/etc/fstab

/root

/usr/local

/var
3.7 Click on “Client”, configure a value for “Soft client quota” if you have very limited storage, probably s
hould modify “File Backup” frequency and other numbers too if storage is very limited.

3.8 If everything is set, you can now start to backup.

(Note: You can use “urbackupclientctl status” command from Proxmox VE host to check UrBackup status)

4 Restore
To restore the Proxmox VE host, you just simply perform following steps

4.1 Install the same Proxmox VE version on the new hard drive

4.2 Install UrBackup client again on Proxmox VE host (Make sure the hostname is the same),
and the client connects to the server

Note: If you are restoring to a different hardware, you need to make sure that the hostname is the same and you need
the tokens in “/usr/local/var/urbackup/tokens” [4], or you can use “Download client for Linux” button
from UrBackup web gui, then the token will be included

4.3 There are two ways to restore file and folders, one is through server web gui, another is via client command with
“urbackupclientctl ….”

If you are using the client command on client, use “urbackupclientctl –help” to check available options,
“urbackupclientctl browse” to browse backups and “urbackupclientctl restore-start” to restore,
then simply use the “Restore” button from UrBackup server web gui to restore files/folders