Incremental backup of VM's

K

kbrault

Renowned Member
May 14, 2012
40
0
71
Hello everyone,

I have a thought here an want to get some feedback before I test it.

Can we perform a differential backup of running Linux VM LVM devices by:

  1. Create a snapshot of the LVM device in the Proxmox OS
  2. Mount the LVM snapshot in the Proxmox OS
  3. Perform a rsync to a mounted LVM on another Linux (Proxmox) server
  4. Un-mount the LVM snapshot in the Proxmox OS
  5. Delete the LVM snapshot in the Proxmox OS

Would this provide a robust backup?

Thanks in advance for your help.
 
macday said:
Hi kbrault,

this is already done...look here: http://www.biodec.com/tools/software/backuppc-lvm

cheers
macday
Click to expand...

We are using a similar solution, but contacting directly the hosts and redirecting rsync on the snapshots on pve.

We use to save also ntfs metadata for windows vm in case of bare metal restore.

We are thinking to publish the scripts soon, if you are interested i can notfy here when is done.

Rob
 
Yes please...what I´m also interessted in is implementing this solution with the good-old rsnapshot which I´m using for backuping Windows and Linux Data.

thank you
mac
 
That looks interesting too but it appears that you have to have a snapshot running all the time.
 
kbrault said:
resoli, would you please post what you have so far for your script?
Click to expand...

At the moment we are using backuppc for all but few vm in two clusters, fifty machines or so in total.

We preferred to have two physical backuppc host; all hosts are configured in the usual way on backuppc.
If an host is a vm, the rsync backup command is redirected on vm using an ssh "forced command" towards the pve host it
is running on. So rsync process runs always on pve host.

Snapshots are taken before and removed after running rsync. For windows host we use the more recent version of ntfs-3g for mounting snapshotted filesystem on pve,
and use xattr mapping feature for backing up ntfs metadata (ACL, short names, etc) in a file which is saved along with vm filesystem.

For windows hosts ACLs (ACLs only) are saved also on the hosts, before snapshotting, and the restore script is able to replay saved acls on single files restored interactively.

rob
 
Last edited:
kbrault said:
That looks interesting too but it appears that you have to have a snapshot running all the time.
Click to expand...

No, snapshots are taken from the host, acting inside the ssh/rsync redirection script; a Sync on traget disk is done before sapshotting, and optionally a set of task (stop/start of services, dump or lock databases ...) is executed, immediately before and after snapshot creation.

Snapshot creation action is launched towards the pve host, customized with a set of parameters (snapshot max dimension, ecc) set by target disk on a configuration file local to vm.

Snapshot is removed right after the end of rsync process.

It is possible to run multiple concurrent backups on the same host, if targeted to different disks.

rob
 
Hi Rob,

After reviewing the readme file https://github.com/mpalmer/lvmsync#readme I can't see how this would work for me unless I have a snapshot running all the time (I do not want to take the performance hit on the host).

Basically for an nightly incremental backup:

  1. Create an initial snapshot
  2. Backup the initial snapshot with dd (this would be a full backup)
  3. Create a new snapshot
  4. Run lvmsync on the new snapshot to transfer the changes since the previous snapshot (this would be an incremental backup)
  5. Delete the previous snapshot and leave the new snapshot running
  6. Every 24 hours repeat steps 3-5. The new snapshot becomes the previous snapshot.
 
macday said:
Really good work Rob...what do you think...would your script work with a KVM-VM with backuppc on the PVE-Host (the pool would be on an NFS-Share)..thanks
Click to expand...

Thanks!

I think that a VM as BackupPC host should work, but it is a suboptimal choice; check the network load on PVE during backups using iftop or similar;
Bear in mind that access to the pool filesystem should be as quick as possible.

Finally, i don't know if hardlinking (base of BackuPC pool operations) works on an NFS filesystem.

rob
 
resoli said:
Thanks!

I think that a VM as BackupPC host should work, but it is a suboptimal choice; check the network load on PVE during backups using iftop or similar;
Bear in mind that access to the pool filesystem should be as quick as possible.

Finally, i don't know if hardlinking (base of BackuPC pool operations) works on an NFS filesystem.

rob
Click to expand...

I would stress that using a VM as BackupPC host could be a viable oction, if you take away NFS. The easiest way is to provide a physical dedicated disk directly to the vm for the pool filesystem.

rob
 
resoli said:
Thanks!

I think that a VM as BackupPC host should work, but it is a suboptimal choice; check the network load on PVE during backups using iftop or similar;
Bear in mind that access to the pool filesystem should be as quick as possible.

Finally, i don't know if hardlinking (base of BackuPC pool operations) works on an NFS filesystem.

rob
Click to expand...

Hi Rob,

I´m confused with the wiki. Could you help me.

thanks mac
 
Hello mac

I will be glad to help; which part of the howto on the wiki is unclear?

rob
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back