Backing up Proxmox Host & VMs

[virtualtechie]

I'm looking to backup my Proxmox host and the VMs I have installed. I'm trying to be prepared in the event the drive running as my boot drive crashes, same for the drive running as my VMs storage.

I would like the best way to restore my host and VMs as easy and as fast as possible. I am mostly a Windows guy but since starting to play with Proxmox, It's sparked a huge interest in wanting to learn Linux and become better at it.

Thanks!

 

[mira]

Can you use Proxmox Backup Server [0]? In this case, backing up and restoring your VMs is quite easy and can be done from the PVE GUI [1].

For your host, the most important configs are `/etc/network/interfaces`, `/etc/hosts`, `/etc/hostname` and everything in `/etc/pve` from a PVE standpoint.
If you've installed and configured other software as well, then those configs would probably be important as well.
For those files you could use the proxmox-backup-client [2], or any other backup software.


[0] https://www.proxmox.com/en/proxmox-backup-server
[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_vzdump
[2] https://pbs.proxmox.com/docs/backup-client.html#creating-backups

 

[Dunuin]

In addition to backing up guests and PVE config files to your Proxmox Backup Server (PBS), you could also backup the whole system disk on block level. This also could be done from CLI after booting into a Debian 11 with the proxmox-backup-client installed, whichwould allow you to backup that whole system disk to your PBS as well. Another option would be to boot into a clonezilla to create a blocklevel backup of the system disk and store it as an image on a NFS/SMB share or dedicated external/internal disk.
But both got the downside that it can't be automated and that your PVE can't be running while doing the backups...so downtime.

 

[alexskysilk]

If you've installed and configured other software as well, then those configs would probably be important as well.
For those files you could use the proxmox-backup-client [2], or any other backup software.

Are there plans to make a bare metal restore available for PBS? that would be pretty neat

 

[Dunuin]

Are there plans to make a bare metal restore available for PBS? that would be pretty neat

Its on the roadmap for a long time:

Roadmap​

• Proxmox VE host backup

I guess it's not that easy as PVE isn't an appliance. And I would bet I couldn't use it with my full system encrypted PVE, as it is too much customized.
A proxmox-backup-client ISO would be nice with some UI/GUI to to backup and restore whole disks and single partitions to/from PBS.

 

[virtualtechie]

Can you use Proxmox Backup Server [0]? In this case, backing up and restoring your VMs is quite easy and can be done from the PVE GUI [1].

For your host, the most important configs are `/etc/network/interfaces`, `/etc/hosts`, `/etc/hostname` and everything in `/etc/pve` from a PVE standpoint.
If you've installed and configured other software as well, then those configs would probably be important as well.
For those files you could use the proxmox-backup-client [2], or any other backup software.


[0] https://www.proxmox.com/en/proxmox-backup-server
[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_vzdump
[2] https://pbs.proxmox.com/docs/backup-client.html#creating-backups

Thanks for the info

 

[virtualtechie]

In addition to backing up guests and PVE config files to your Proxmox Backup Server (PBS), you could also backup the whole system disk on block level. This also could be done from CLI after booting into a Debian 11 with the proxmox-backup-client installed, whichwould allow you to backup that whole system disk to your PBS as well. Another option would be to boot into a clonezilla to create a blocklevel backup of the system disk and store it as an image on a NFS/SMB share or dedicated external/internal disk.
But both got the downside that it can't be automated and that your PVE can't be running while doing the backups...so downtime.

Thank for the info

 

[alexskysilk]

I guess it's not that easy as PVE isn't an appliance.

Ironically, this wasn't the use case I had in mind; from my point of view, proxmox hosts can and should be interchangeable and really need no backups. I was more thinking of making vm and metal deployments interchangeable, eg backup a vm and restore to a server, or vice versa. The roadmap does not stipulate this as a usecase but it should be doable once the bootable restoration client is done.

 

[Dunuin]

Ironically, this wasn't the use case I had in mind; from my point of view, proxmox hosts can and should be interchangeable and really need no backups. I was more thinking of making vm and metal deployments interchangeable, eg backup a vm and restore to a server, or vice versa. The roadmap does not stipulate this as a usecase but it should be doable once the bootable restoration client is done.

At least here replacing a node isn't that easy. I need to install NUT client/server for UPS shutdown, Zabbix client for monitoring, filebeat to send logs to remote log server, setup stuff for PCI passthrough, customize the initramfs with additional tools for remote unlocking of encrypted root system and VLAN/bond support, edit the sysctl.conf, install a lot of packages and configure a bunch of stuff for zabbix templates, add a lot of custom systemd services to do stuff like PVE config backups to PBS, ...
And even if PBS could do PVE host backups, I think I would still need to do all the above again.

 

[alexskysilk]

At least here replacing a node isn't that easy. I need to install NUT client/server for UPS shutdown, Zabbix client for monitoring, filebeat to send logs to remote log server, setup stuff for PCI passthrough, customize the initramfs with additional tools for remote unlocking of encrypted root system and VLAN/bond support, edit the sysctl.conf, install a lot of packages and configure a bunch of stuff for zabbix templates, add a lot of custom systemd services to do stuff like PVE config backups to PBS, ...
And even if PBS could do PVE host backups, I think I would still need to do all the above again.

How fantastic- this should make a perfect case study for what I mean.

NUT server should not be installed on the host. Client, zabbix, filebeat can and probably should be ansibled/scripted as part of your node setup process.same should apply to customize the initramfs with additional tools for remote unlocking of encrypted root system and VLAN/bond support, edit the sysctl.conf (dont do that, just copy what you need to sysctl.d), install a lot of packages and configure a bunch of stuff for zabbix templates, add a lot of custom systemd services to do stuff like PVE config backups to PBS, yadda yadda. I have similar loadouts on my nodes and is a one touch process.

pci passthrough- that would probably have to be done by hand anyway unless you're reinstalling on the identical server. I'd probably still do it manually even then although I dont do anything that requires it in my env.

Moral of the story- put all your custom stuff in a setup script. no backups required

 

[Dunuin]

I wrote some setup scripts to make it easier. But it really depends on how much nodes you are installing per year. If it is just one or 2 nodes, its more work to create and maintain those scripts then just do the install. Some repos changed, some zabbix templates weren't supported anymore, for stuff like IPMI monitoring you need a custom template for each mainboard model, ...
And installing a PVE with full system encryption is an adventure on its own.

 

[RickWohleber]

I'm bumping this as I haven't found a pertinent active thread (yet). Any word on when we can do a bare metal restore of a PVE Host (not the VM's, but rather the Host itself)? This would be invaluable. Obviously, to make full use of this we'd need to see a "Recover from PBS" option in the Installation process when we do a reinstall....

 

[proxfire44]

I'm bumping this as I haven't found a pertinent active thread (yet). Any word on when we can do a bare metal restore of a PVE Host (not the VM's, but rather the Host itself)? This would be invaluable. Obviously, to make full use of this we'd need to see a "Recover from PBS" option in the Installation process when we do a reinstall....

I am also interested in this. I'm surprised there isn't more guidance on this topic.

For Proxmox hosts using ZFS for the boot drive, I'm curious if a simple ZFS snapshot would be sufficient.

 

[alexskysilk]

Any word on when we can do a bare metal restore of a PVE Host (not the VM's, but rather the Host itself)?

you can always use clonezilla if this is useful to you.

 

[fletch]

you can always use clonezilla if this is useful to you.

Have you ever been successful with clonezilla? It would always fail for me and could not complete. Prox is the only thing I have seen it fail on to date.

 

[alexskysilk]

Have you ever been successful with clonezilla?

Funny you should ask. a few months ago I was set to replace a couple of nodes in a slow spinning ceph cluster. Under normal circumstances I'd just yank the OSDs from the cluster, replace the node, and reinstall the drives in the replacement node- but with spinning (large) drives, this process is very time consuming so I decided to do what I normally wouldnt (see my comments further up the thread)- take a standing proxmox install and move it to another box.

I didnt want to do this without a way to fall back should anything went sideways (since it SO OFTEN does) so I used clonezilla to do exactly what you ask- to further complicate things, proxmox was installed on a zfs mirror. Except for some fiddling with the bootloader on the destination, it worked like a charm and saved me a ton of time and headache.

In general, I have never had any trouble using clonezilla; its a pretty simple tool- anything you can dd you can copy.

 

[fletch]

Funny you should ask. a few months ago I was set to replace a couple of nodes in a slow spinning ceph cluster. Under normal circumstances I'd just yank the OSDs from the cluster, replace the node, and reinstall the drives in the replacement node- but with spinning (large) drives, this process is very time consuming so I decided to do what I normally wouldnt (see my comments further up the thread)- take a standing proxmox install and move it to another box.

I didnt want to do this without a way to fall back should anything went sideways (since it SO OFTEN does) so I used clonezilla to do exactly what you ask- to further complicate things, proxmox was installed on a zfs mirror. Except for some fiddling with the bootloader on the destination, it worked like a charm and saved me a ton of time and headache.

In general, I have never had any trouble using clonezilla; its a pretty simple tool- anything you can dd you can copy.

I tried with the latest version tonight and still gives me errors. I have mine set to check the image to be sure it can be restored and that is where it fails every time. Maybe it can really be restored but I don't want to take any chances so I am going to assume that I am upgrading to 8 with no chance of recovery if something goes wrong. Thanks.

 

[alexskysilk]

I tried with the latest version tonight and still gives me errors.

Dont be so quick to dismiss those errors as a problem with clonezilla; they are likely trying to tell you something important. What do those errors say?

 

[fletch]

That is exactly what I mean - I can't trust the backup. What it said was it could not check some of the partitions and then the check aborted. If I had a spare drive, I could see if it would actually restore but right now I don't. I sure wish proxmox had a way to backup the host.

 

[fletch]

UPDATE : Clonezilla worked - mostly.

I kept the backup with errors. I bought a new drive for the node and attempted a restore. It worked though I did have an id mismatch with LVM -thin. Not sure if this was due to the errors I got from Clonezilla or because I let it expand the drive (went from 256g to 1T). I was able to fix this per instructions here on this forum. Knowing I had a backup that worked, I finally upgraded proxmox from 7 to 8 last night! Yea now just need to learn this stuff better or get a backup node.