VM Incremental backup for 3-party solution

radekk

Member
Feb 19, 2021
6
0
6
49
Hello,

I'm looking for some resources or documentation how to implement an incremental VM backup using 3-party backup solution in the same way or similar as PBS do.
Thanks in advance.

Radek
 
That heavily depends on your "Third Party Solution". In general we have our own backup implementation directly in QEMU for both VMA and PBS, both are controlled by "vzdump" and related APIs from our perl code. Adding another solution is certainly not an easy undertaking.

You might get away without integrating into our stack too deeply if you manage your backups entirely externally from PVE, but you'd probably still need QEMU support to be able to take consistent backups (though I have heard some people just taking ZFS snapshots as backups, so ignoring QEMU entirely is an option if you don't value consistency too highly).

If you're not discouraged though, we appreciate all contributions :)

Check our current implementation on our git: https://git.proxmox.com/
The most interesting repositories will probably by pve-qemu, proxmox-backup-qemu and qemu-server (especially VZDump.pm).
 
  • Like
Reactions: Sebastian Schubert
Thanks @Stefan_R for your promptly answer.

In the recent years I've integrated Proxmox backups using vzdump in the 3-party backup solution - Bacula Enterprise - and it is working great.
I have a broad knowledge of the VM backup solutions as recently I've integrated the Block Level Snapshot-Based Incremental backups for XenServer using official API in the same backup product. I fully understand that it would be not an easy task especially when, as I understand it, no official API was designed. I'm not very familiar with your low-level backup code and I'm seeking some references and hints in these area. I'll appreciate any help as your git project repository is huge.

Radek
 
I suggest taking a look at the projects I've mentioned above, as well as our developer documentation. For specific technical questions it is advisable to ask them on our pve-devel mailing list instead, we'd be glad to explain pieces of our code there if you need assistance (see the link in the developer docs).

Without concrete questions, I can't give you concrete answers :) Our pve-qemu code base is not that big, most of it uses upstream QEMU primitives together with our rust code linked in from proxmox-backup-qemu, so that might be a starting point.

Additionally, if your solution already has QEMU support, it might not even be necessary to patch anything, just ignore the PVE part and backup the QEMU instances directly.
 
Thanks. I'll check your resources and will ask concrete questions on pve-devel mailing list for sure.

if your solution already has QEMU support, it might not even be necessary to patch anything, just ignore the PVE part and backup the QEMU instances directly

What do you mean by "QEMU Support" in this case?

best regards
Radek
 
What do you mean by "QEMU Support" in this case?
If your backup solution can already backup virtual machines running under KVM/QEMU, you may not need to touch PVE code. PVE uses QEMU as it's underlying hypervisor, so anything that supports that will work on PVE VMs. QEMU is fairly widespread in virtualization, outside of PVE too (see libvirt for example).

Here's some documentation from upstream QEMU for example, on how they envision backups - many of the techniques discussed there are also in use by our own backup code: https://qemu.readthedocs.io/en/latest/interop/bitmaps.html
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!