Please help me understand a few things about OpenVZ backups

[drzoidberg33]

Hi,

I'm busy deploying a few containers to a Proxmox machine and I am busy configuring live backups, I've scoured the forums and everything is working so far but one thing I'm still unsure about it the amount of free space that is required on the volume group. Must the free space be equivalent to the size of the container being backed up or equivalent to the amount of memory the container has?

I currently have 4 volume groups, 1 which is the default and 3 others I created. The default install left 16GB in the pve VG so I did the same with mine. Here is the output of vgs:

root@proxmox0:/mnt/disk1/dump# vgs  VG   #PV #LV #SN Attr   VSize   VFree 
  pve    1   3   0 wz--n- 232.38g 16.00g
  vg1    1   1   0 wz--n- 931.51g 16.00g
  vg2    1   1   0 wz--n- 931.51g 16.00g
  vg3    1   1   0 wz--n- 931.51g 16.00g

And lvs:

root@proxmox0:/mnt/disk1/dump# lvs
  LV   VG   Attr      LSize   Pool Origin Data%  Move Log Copy%  Convert
  data pve  -wi-ao--- 143.39g                                           
  root pve  -wi-ao---  58.00g                                           
  swap pve  -wi-ao---  15.00g                                           
  lv1  vg1  -wi-ao--- 915.51g                                           
  lv2  vg2  -wi-ao--- 915.51g                                           
  lv3  vg3  -wi-ao--- 915.51g

Here is my vzdump.conf:

root@proxmox0:/mnt/disk1/dump# cat /etc/vzdump.conf
# vzdump default settings


#tmpdir: DIR
#dumpdir: DIR
#storage: STORAGE_ID
#mode: snapshot|suspend|stop
#bwlimit: KBPS
#ionice: PRI
#lockwait: MINUTES
#stopwait: MINUTES
size: 16000
#maxfiles: N
#script: FILENAME
#exclude-path: PATHLIST

Is this configuration okay and will I be able to do live backups when the containers get larger? Currently the biggest one I have is only around 9GB, here is the output from that backup:

INFO: starting new backup job: vzdump 100 101 --quiet 1 --mailto it@mydomain.com --mode snapshot --compress lzo --storage disk1
INFO: Starting Backup of VM 100 (openvz)
INFO: CTID 100 exist mounted running
INFO: status = running
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating lvm snapshot of /dev/mapper/vg1-lv1 ('/dev/vg1/vzsnap-proxmox0-0')
INFO:   Logical volume "vzsnap-proxmox0-0" created
INFO: creating archive '/mnt/disk1/dump/vzdump-openvz-100-2014_02_11-19_42_02.tar.lzo'
INFO: Total bytes written: 11935180800 (12GiB, 4.8MiB/s)
INFO: archive file size: 9.70GB
INFO: Finished Backup of VM 100 (00:40:48)
INFO: Starting Backup of VM 101 (openvz)
INFO: CTID 101 exist mounted running
INFO: status = running
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating lvm snapshot of /dev/mapper/vg1-lv1 ('/dev/vg1/vzsnap-proxmox0-0')
INFO:   Logical volume "vzsnap-proxmox0-0" created
INFO: creating archive '/mnt/disk1/dump/vzdump-openvz-101-2014_02_11-20_22_50.tar.lzo'
INFO: Total bytes written: 2657761280 (2.5GiB, 34MiB/s)
INFO: archive file size: 1.68GB
INFO: Finished Backup of VM 101 (00:01:31)
INFO: Backup job finished successfully
TASK OK

Also, is there any reason why the first backup was so slow? 5MB/sec for the first then 34MB/sec on the second. Both are just backing up to a local disk at this stage.

TIA

 

[e100]

The size setting in vzdump controls the size of the snapshot on the LVM volume.

Snapshots perform copy on write (CoW) which means the old data is copied to the snapshot when new dats is written to the volume.

Your snapshot size needs to be larger than the amount of data that will be written to the volume while the backup is taking place.

Eg. If your server is writing 1MB/sec to the volume and it takes 5 minutes (300s) to backup, then your snapshot size needs to be larger than 300MB.

Typically you will have multiple openvz containers on a single filesystem which resides on a single volume. So you need to account for the writes in all of the containers on that volume not just the container being backed up.

Your VM backup sizes were very different which is likely why the second (smaller) backup was faster. The smaller archive likely fit in the write buffer better thus appeared to"write" faster.

 

[drzoidberg33]

The size setting in vzdump controls the size of the snapshot on the LVM volume.

Snapshots perform copy on write (CoW) which means the old data is copied to the snapshot when new dats is written to the volume.

Your snapshot size needs to be larger than the amount of data that will be written to the volume while the backup is taking place.

Eg. If your server is writing 1MB/sec to the volume and it takes 5 minutes (300s) to backup, then your snapshot size needs to be larger than 300MB.

Typically you will have multiple openvz containers on a single filesystem which resides on a single volume. So you need to account for the writes in all of the containers on that volume not just the container being backed up.

Your VM backup sizes were very different which is likely why the second (smaller) backup was faster. The smaller archive likely fit in the write buffer better thus appeared to"write" faster.

Thank you for the explanation, it makes sense now. So I should be good with 16GB free then.

The first backup that I did contains a folder with thousands of small files, I'm thinking this may be the reason it is so much slower than the other CT.

Thanks again for laying this out to a noob

 

[e100]

Thank you for the explanation, it makes sense now. So I should be good with 16GB free then.

The first backup that I did contains a folder with thousands of small files, I'm thinking this may be the reason it is so much slower than the other CT.

Thanks again for laying this out to a noob

16GB is likely fine based on the info you have provided.

You are likely right, reading many small files will slow things down.