Backup KVM with stop + shapshot

[giner]

Hello,

What do you think about backup mode such as "stop + snapshot"?
1. Stop allows backup to be consistent.
2. Snapshot allows to make backup faster.

Steps:
1. Stop VM.
2. Make Snapshot.
3. Start VM.
4. Do backup.
5. Remove Snapshot.

Any thoughts?

 

[dietmar]

interesting idea - thought I don't know if that would be used by many people.

 

[giner]

interesting idea - thought I don't know if that would be used by many people.

I have a KVM machine with WSUS/Antivirus/Somthing. Disk usage is about 100Gb. This machine have enabled write cache for better performance and backuping by snapshot is risky to lose data.
Suspend and Stop modes take a lot of time for this KVM. From logs:

1. Snapshot.
2. Sync (about 4 hours) - unneded operation for KVM. it is useful only for OpenVZ.
3. Stop/Suspend.
4. Backup (about 4 hours).
5. Remove snapshot.

It's too long.

Stop/Suspend and then snapshot would be faster.

Have a nice day,
Stas

 

[giner]

Hello dietmar,

What do you think about these changes?

diff -ur vzdump_2009-12-22.orig/OpenVZ.pm vzdump_2009-12-22/OpenVZ.pm
--- vzdump_2009-12-22.orig/OpenVZ.pm	2009-12-22 15:00:10.000000000 +0300
+++ vzdump_2009-12-22/OpenVZ.pm	2010-01-05 11:27:50.129753826 +0300
@@ -197,7 +197,7 @@
 
     my $hostname = hostname(); 
 
-    if ($mode eq 'snapshot') {
+    if ($mode eq 'snapshot' || $mode eq 'stop') {
 
 	my $lvmmap = PVE::VZDump::get_lvm_mapping();
 	my ($srcdev, $lvmpath, $lvmvg, $lvmlv, $fstype) =

Stas

 

[dietmar]

What do you think about these changes?

Sorry, I do not understand the reasoning behind that patch. Please can you explain?

 

[dietmar]

Oh, you want to implement "stop + snapshot" mode? If so, you can't simply change the 'stop' mode into "stop + snapshot" - instead we need to introduce an new mode, and I guess this is a big change.

 

[giner]

Hello,

What do you think about backup mode such as "stop + snapshot"?
1. Stop allows backup to be consistent.
2. Snapshot allows to make backup faster.

Steps:
1. Stop VM.
2. Make Snapshot.
3. Start VM.
4. Do backup.
5. Remove Snapshot.

Any thoughts?

Oh, you want to implement "stop + snapshot" mode? If so, you can't simply change the 'stop' mode into "stop + snapshot" - instead we need to introduce an new mode, and I guess this is a big change.

You are right.
Is it possible to implement? It will give posibility to make exactly consistent backups with a very little downlime.

 

[dietmar]

You are right.
Is it possible to implement? It will give posibility to make exactly consistent backups with a very little downlime.

Sure - I simply have no time to work on that.

 

[dietmar]

You are right.
Is it possible to implement? It will give posibility to make exactly consistent backups with a very little downlime.

And I also have some doubts if it is really usefull, because stopping a VM is not really reliable (windows is kniown to have problems)

 

[giner]

...windows is kniown to have problems

I also have had, but I have completely solved it. I've watched backup logs for a while (it's being sent me on email) therefore I'm always convinced that it works as good as needed.

Now all of my machines are backing up using "snapshot" mode on Tuesday and by "stop mode" on Saturday. Because snapshot mode insufficiently reliable while a guest machine uses write cache or runs any kind of database engine.

 

[giner]

And I also have some doubts if it is really usefull, because stopping a VM is not really reliable (windows is kniown to have problems)

I'd like to set eyes on this problem again.
When I do snapshot backup up without stopping machine I have so called "crash consistent backup" when files system is consistent but some applications probably didn't save all their data.
For example: mail server which is working on virtual machine have received email message and is going to proccess it. It can be spam filter or post-convertion or extracting attaches or some else. Backup can occur while proccess haven't yet done and some data will be lost. In another case when I stop machine all services are going to be stoped to and no data will be lost. It's true for databases and many other applications too.

 

[vlad]

I do this by hand:
- stop kvm with windows
- make snapshot
- start windows kvm
- attach snapshot to a kvm with linux
- start linux kvm and run ntfsclone
- stop linux kvm
- remove snapshot
I think images are smaller that way and backup is reliable. I wish there would be an automated process for this.

 

[giner]

I do this by hand:
- stop kvm with windows
- make snapshot
- start windows kvm
- attach snapshot to a kvm with linux
- start linux kvm and run ntfsclone
- stop linux kvm
- remove snapshot
I think images are smaller that way and backup is reliable. I wish there would be an automated process for this.

Could you explain what the reason to use another linux for running ntfsclone? Looks like it can be produced on the main host.

 

[vlad]

Could you explain what the reason to use another linux for running ntfsclone? Looks like it can be produced on the main host.

Internal partitions of snapshot are not visible on the host unless you install multipath tools or ntfsclone wont work on raw drive. I dont want to sabotage hosts reliability by installing multipath tools, in past those would create strange problems where I could not unmount/remove snapshots.

 

[giner]

Internal partitions of snapshot are not visible on the host unless you install multipath tools or ntfsclone wont work on raw drive. I dont want to sabotage hosts reliability by installing multipath tools, in past those would create strange problems where I could not unmount/remove snapshots.

Sorry, I don't quite see what you mean. I can do snapshot and then just copy image file (raw or qcow2) to another place.

 

[vlad]

Sorry, I don't quite see what you mean. I can do snapshot and then just copy image file (raw or qcow2) to another place.

I don't use image files because those should be slowing down guest machines, I use LVMs instead, for optimal speed.

 

[giner]

I don't use image files because those should be slowing down guest machines, I use LVMs instead, for optimal speed.

Of course. I mean you can stop VM, create LVM snapshot, start VM, mount snapshot to some directory, copy VM images to backup place, unmount and remove snapshot.

 

[vlad]

Of course. I mean you can stop VM, create LVM snapshot, start VM, mount snapshot to some directory, copy VM images to backup place, unmount and remove snapshot.

In my case, you can't do that. There are no VM images(no raw and no qcow), KVMs use whole LVM block devices, i.e. /dev/pve/vm-102-disk-1

 

[giner]

In my case, you can't do that. There are no VM images(no raw and no qcow), KVMs use whole LVM block devices, i.e. /dev/pve/vm-102-disk-1

lvcreate --size 1G --snapshot --name snap /dev/pve/vm-102-disk-1
dd if=/dev/pve/snap of=/mnt/backup/vm-102-disk-1-image bs=1M
lvremove /dev/pve/vm-102-disk-1

Can you do this way?

 

[vlad]

lvcreate --size 1G --snapshot --name snap /dev/pve/vm-102-disk-1
dd if=/dev/pve/snap of=/mnt/backup/vm-102-disk-1-image bs=1M
lvremove /dev/pve/vm-102-disk-1

Can you do this way?

Yes, but its a waste of backup space if there is not much data. What I need to do is:
ntfsclone -s -o backupfile.ntfs /dev/pve/snap0p1
or
ntfsclone -s -o backupfile.ntfs /dev/pve/snap0p5
where snap0p1 primary partition in KVMs drive and snap0p5 is a secondary.

if vm-102-disk-1 is 100gb, and snap0p1 uses only 15gb, snap0p5 using only 25gb then my backups will be only 40gb in size and your backups will be whole 100gb, do you understand now?