Split VM backup to multiple datastores for different disks

[M1k3y]

Hello,

I'm cureently in need of a very particular backup configuration. I need to backup different disks of the same VM to different backup targets.

Is there a way to either:

- modify the target datastore for a specific disk
- exclude a disk from a specific backup job only
- modify the target namespace of a specific disk

Any of these options would solve my current problem.

Alternatively, a way to exclude certain disks from the tape backup job in pbs would also get me most of the way, although one of the other solutions would be preferred.

I'd prefer a solution that can be configured from the UI, but I'm ready to dig into the configuration files if needed. I would also much prefer to not employ external tools for this.

 

[bbgeek17]

You may want to ask in the sister forum https://forum.proxmox.com/forums/proxmox-backup-installation-and-configuration.24/

AFAIK:
1) No
2) No
3) No

That is - there is no persistent configuration checkmark that you can set from GUI to achieve what you want.

You could employ hook scripts and modify the VM/backup configuration on the fly prior to specific job starting.
Keep in mind that you will loose ability to restore your VM with a single click or in-place. Trying to restore disk1 will remove existing disk2, if it exists. You will likely have to restore each part to its own new VM and then move the disks around to merge them back.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Kingneutron]

> I need to backup different disks of the same VM to different backup targets

Why is the pertinent question here?

If the 2nd / data disk is overly large or something, you may want to think outside of the box for this and move whatever it contains to a shared drive / outside the VM - so the attached disk gets backed up with the rest of the VM as intended, but doesn't use a whole lot of space.

 

[M1k3y]

Yes, splitting out the data to another VM is something I'm considering. However I was hoping for a solution that would not require it as the data is specific to this VM only and does not get accessed outside of it directly. Also I wanted to prevent the additional overhead this solution would incure. Also to make this approach work perfectly, I would need define dependencys between VMs which seems to be not supported by proxmox.

 

[Kingneutron]

If it's a win10 vm, you could use Veeam (bare-metal backup/restore) free agent to just backup the C: drive to a Samba share. Then you could make a separate backup job for the D: drive to different storage / different schedule. (Or even use something like rsync)

If Linux vm, you could use tar or fsarchiver (or Clonezilla, etc) to just backup the root filesystem.

In either case, you could turn off backups at the PVE host level and just treat the VM like any other instance, run backup from inside it - taking only what you need to restore the OS. If the OS drive doesn't change much, you could set it to backup weekly or even monthly.

It's easy to get "stuck" thinking of things only at the host level; remember you're running a full OS in a VM, so you have multiple backup options at the guest level as well.

Just remember to test your restores - a backup is useless if restoring it doesn't get you back to your intended full-environment experience.

 

[Dunuin]

I would need define dependencys between VMs which seems to be not supported by proxmox.

I'm also missing built-in dependency management but you could work around that by making use if start order + start delay. And if you really got too much time you could even write a script for orchestration via PVE/PBS API.

 

[VictorSTS]

You may try something that may work but feels like an ugly risky hack that I would never use in production... writing this out of my mind, I may be miss some step.

Let's suppose the running VM vmid250 has 3 drives and you want 3 different schedules to 3 different backup storages, each with their own prune policy.

- Copy the vm config file to two other files:

cd /etc/pve/qemu-server/
cp 250.conf 2501.conf
cp 250.conf 2502.conf

- From webUI or editing the file, modify the config of each one to include one drive each.
- Change the boot order of the VMs 2501 and 2502 to network o CD, so they will never boot the installed OS.
- Place each VM in a backup task to the appropriate backup storage.
- Keep all VMs in the same PVE host the will never be backed up simultaneously.

Now that I'm thinking about this again, I wouldn't do this because the QEMU thread running the VM and writing to the disks has no way to know that the disk is being backed up, so no proper way to do the snapshot before the backup. Also not sure that the source disk will be safe as another QEMU instance will try to snapshot it while in use.

IMHO, backup hook scripts seem the safest way to implement your requirement atm.

I would need define dependencys between VMs which seems to be not supported by proxmox.

Besides the already mentioned start order, you can also use hook scripts for VMs [1]. Used it to make sure DB server is up before starting the app server, waiting for the DB to be fully up.

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_hookscripts

 

[M1k3y]

In either case, you could turn off backups at the PVE host level and just treat the VM like any other instance, run backup from inside it - taking only what you need to restore the OS. If the OS drive doesn't change much, you could set it to backup weekly or even monthly.

It's easy to get "stuck" thinking of things only at the host level; remember you're running a full OS in a VM, so you have multiple backup options at the guest level as well.

Thanks for the suggestions. I didn't even think about the possibility of backing up the relevant storage via the guest OS. That is a really neat option. I will backup the VM itself and the OS disk from PVE and the data disks from inside the VM. This way, I can restore the machine easily, even if I have to rebuild the host completely from scratch.

Just remember to test your restores - a backup is useless if restoring it doesn't get you back to your intended full-environment experience.

Yes, that is something I'm aware of. Luckily I have a second, currently unused server available to me. I'm planning a full disaster recovery test in the next days, assuming I have nothing else than my tape backups (Not even my Laptop. I'll go with a blank fresh Linux VM for this. Don't want to accidentally rely on any tools, documentation or credentials that might be inaccessible in this scenario).

I'm also missing built-in dependency management but you could work around that by making use if start order + start delay. And if you really got too much time you could even write a script for orchestration via PVE/PBS API.

Yeah. Some dependency management would be nice. Doesn't need to be something fancy. A simple "VM-A needs VM-B running" would be enough. Though an option to probe a TCP Port or a simple HTTP Healthcheck to ensure VM-B has it's services running would be nice as well. Heck, you could make it a bash command or script and use its exit code for the check.

You may try something that may work but feels like an ugly risky hack that I would never use in production... writing this out of my mind, I may be miss some step.

[...]

Now that I'm thinking about this again, I wouldn't do this because the QEMU thread running the VM and writing to the disks has no way to know that the disk is being backed up, so no proper way to do the snapshot before the backup. Also not sure that the source disk will be safe as another QEMU instance will try to snapshot it while in use.

Nice Idea, but that's a bit to hacky for my liking. My main concern here is that someone else should be able manage the setup as long as they have some experience with PVE and PBS without assistance. Such "hidden" hacks are therefor of the table at the moment.

Besides the already mentioned start order, you can also use hook scripts for VMs [1]. Used it to make sure DB server is up before starting the app server, waiting for the DB to be fully up.

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_hookscripts

Hook scripts look interesting. I'll read into it, but I don't think they are the correct solution for this problem. But I thank you for making me aware of them.