Proxmox host backup and snapshots with timeshift. Or any other such easy tool?

D

dominikp

Member
Aug 28, 2018
36
3
13
39
On my company we use Timeshift for doing snapshots of our Ubuntu/Mint workstations. Its extremely easy to use tool and rescued our workers machines few times without a problems. Just fire up any live cd with timeshift (mint), point where are backups stored, choose latest copy, restore and reboot. Thats it.
TLDR;
Has anyone tried it with proxmox host?

Full version.
I have a client where i set up the proxmox machine. I moved their services from one physical machine with Ubuntu server to few separate VMs/CTs on proxmox. And i would like to give them some easy tool to restore this server in case of some crash or something. The way Timeshift works would be perfect.
 
BobhWasatch said:
Is there a reason you can't just use Timeshift?
Click to expand...

Yes, lack of any information about it. Offcourse i plan to test this setup on some (non-prod)machine, but first i wanted to know about any quirks/problems i could expect or for what kind of things i should be most careful. Workstations are one thing but hypervisor is a little bit different.
 
I think I misunderstood your question. For some reason I thought you were talking about restoring a VM, not the whole server. Sorry about that.

The "official" way is to use the built-in vzdump backup. Disaster recovery is to re-install Proxmox and then restore the VM's from those backups. That works well but backups can take a long time and there's no incremental backup to help with that. It is also not possible to restore individual files. On the other hand it allows you to quickly rebuild a system from scratch without any extra tools.

So most people do vzdump at a relatively longer interval and use another method inside the VM's in between vzdump backups. That makes an extra step to recover from a dead host, but it allows restoring individual files and the like for the more common use-cases. Your existing Timeshift method might fit well in that role and would work pretty much the same as it does on real machines.

One issue with using a third-party backup system on the Proxmox host itself will be that the VM's are "live" and making changes to their disk images during the backup. So you might have to do some scripting to freeze/snapshot/unfreeze them in order to get a consistent backup. That is what the built-in vzdump backups do. Even then there can be issues with things like databases that do application-level caching. Solutions to that are necessarily application-specific.

Backup is a complicated topic :cool:
 
I think I maybe experiencing issues with timeshift run on the hosts. I have 2 hosts as cluster with my workstation running qnetd. Maybe sure your quorum is good for this.

Run `df` on your host. Notice /etc/pve ?

Note that /etc/pve is mounted using fuse, and I think this maybe a ceph shared cluster folder. Normally, timeshift is pretty good at stopping at the filesystem boundary. And my data folders are intentional excluded from timeshift backups (automatically, I think due to being mounted in /mnt is my guess) In my case it backed up /etc/pve to regular folder in filesystem of host. Which I think is maybe interfering with startup fuse wants to mount /etc/pve probably into an empty folder? Because I restored inside "safe mode" prior to mounting /etc/pve I think I will rename /etc/pve and create a new /etc/pve in the hopes this mount will sync up next reboot.

```
pwd
/timeshift/snapshots/2025-02-07_19-28-54/localhost
root@hulk:/timeshift/snapshots/2025-02-07_19-28-54/localhost# du -d1 -h
4.0K ./sandisk_zfs
6.1M ./etc
20K ./home
4.0K ./media
4.0K ./srv
4.0K ./proc
96M ./boot
4.0K ./mnt
4.0K ./tmp
4.0K ./sys
4.0K ./run
1.3G ./var
3.3G ./usr
4.0K ./root
4.0K ./timeshift
4.0K ./dev
4.0K ./opt
4.8G .
```
So.... after a failed restore using timeshift, I am about to do this and reboot:

```
mv /etc/pve /etc/pve-DISABLED
mkdir /etc/pve
reboot
```
If this works I think I will add /etc/pve to the exclude filter. Also... perhaps timemachine is not ideal for backing up proxmox host? It has worked to save my (home) cluster before mind. While I don't see entries in fstab I think this is mounting from some other config. I hope I am correct. Not sure if this is done with Ceph. I did my bad restore in "Control-D" recovery mode on machine that has gone AWOL. I think my successful restores were done on a machine with /etc/pve mounted R/W at the time (and thus they went OK amazingly).

PS I am not the most experienced, it's possible my advice is bad? Also I am a bit weary of what happened with /boot/efi on the cluster node that is giving me issues; but it still boots after doing this problematic restore so hopefully /etc/pve will mount shortly after I hit reply to this forum and give this a shot.
 
Excellent. I can confirm that doing just `mv /etc/pve /etc/pve-DISABLED` and a reboot fixed my cluster. I did not even do `mkdir /etc/pve`. After reboot /etc/pve is being mounted magically. That is the shared config folder. So far so good.
 
Last edited:
After a brief wobble I think it's up for good:

Code: 
root@elite:~# journalctl -f 
May 09 02:43:11 elite pve-ha-lrm[1502]: unable to write lrm status file - unable to open file '/etc/pve/nodes/elite/lrm_status.tmp.1502
' - Device or resource busy
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: members: 1/1082, 2/1423
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: starting data syncronisation
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: received sync request (epoch 1/1082/00000004)
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: received all states
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: leader is 2/1423
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: synced members: 2/1423
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: waiting for updates from leader
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: update complete - trying to commit (got 2 inode updates)
May 09 02:43:13 elite pmxcfs[1082]: [dcdb] notice: all data is up to date

Mite need to delete a lock file for now it seems to be sweet.
 
_gabriel said:
You can't as /etc/pve is mount point filesystem for database stored data.
So can't be restored offline this way.

https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)
Click to expand...
Yeah - this was not done while /etc/pve was mounted. I could not see it in `df` . Timeshift had "copied" from the net-backup a folder-full of configs which did prevent proxmox from continuing the boot (I was not getting to login at that node).

My issue was that the tool timeshift did "restore" the host sans-network and rebuilt this folder as a regular ext4 (LVM?) folder! Whoops. Thankyfully, my terrible hack worked. Suffice to say the machine had sufficient config to connect back into the cluster to rebuild it / network mount it perhaps better description... best to purchase Proxmox Backup Server I would think. Mine is home lab. If used for business, purchase a support license. :)

1746715906588.png

To be fair to `timeshift` it's pretty amazing my cluster is still up and running, so props to the makers of that software. In future I will exclude /etc/pve from timeshift snapshots. It is quite the brute force method I just deployed.
 
You must log in or register to reply here.
Top Bottom