Backup KVM with stop + shapshot

G

giner

Member
Oct 14, 2009
239
0
16
41
Tokyo
Hello,

What do you think about backup mode such as "stop + snapshot"?
1. Stop allows backup to be consistent.
2. Snapshot allows to make backup faster.

Steps:
1. Stop VM.
2. Make Snapshot.
3. Start VM.
4. Do backup.
5. Remove Snapshot.

Any thoughts?
 
Last edited:
dietmar said:
interesting idea - thought I don't know if that would be used by many people.
Click to expand...

I have a KVM machine with WSUS/Antivirus/Somthing. Disk usage is about 100Gb. This machine have enabled write cache for better performance and backuping by snapshot is risky to lose data.
Suspend and Stop modes take a lot of time for this KVM. From logs:

1. Snapshot.
2. Sync (about 4 hours) - unneded operation for KVM. it is useful only for OpenVZ.
3. Stop/Suspend.
4. Backup (about 4 hours).
5. Remove snapshot.

It's too long.

Stop/Suspend and then snapshot would be faster.

Have a nice day,
Stas
 
Last edited:
Hello dietmar,

What do you think about these changes?

Code: 
diff -ur vzdump_2009-12-22.orig/OpenVZ.pm vzdump_2009-12-22/OpenVZ.pm
--- vzdump_2009-12-22.orig/OpenVZ.pm	2009-12-22 15:00:10.000000000 +0300
+++ vzdump_2009-12-22/OpenVZ.pm	2010-01-05 11:27:50.129753826 +0300
@@ -197,7 +197,7 @@
 
     my $hostname = hostname(); 
 
-    if ($mode eq 'snapshot') {
+    if ($mode eq 'snapshot' || $mode eq 'stop') {
 
 	my $lvmmap = PVE::VZDump::get_lvm_mapping();
 	my ($srcdev, $lvmpath, $lvmvg, $lvmlv, $fstype) =

Stas
 
Oh, you want to implement "stop + snapshot" mode? If so, you can't simply change the 'stop' mode into "stop + snapshot" - instead we need to introduce an new mode, and I guess this is a big change.
 
giner said:
Hello,

What do you think about backup mode such as "stop + snapshot"?
1. Stop allows backup to be consistent.
2. Snapshot allows to make backup faster.

Steps:
1. Stop VM.
2. Make Snapshot.
3. Start VM.
4. Do backup.
5. Remove Snapshot.

Any thoughts?
Click to expand...

dietmar said:
Oh, you want to implement "stop + snapshot" mode? If so, you can't simply change the 'stop' mode into "stop + snapshot" - instead we need to introduce an new mode, and I guess this is a big change.
Click to expand...

You are right.
Is it possible to implement? It will give posibility to make exactly consistent backups with a very little downlime.
 
giner said:
You are right.
Is it possible to implement? It will give posibility to make exactly consistent backups with a very little downlime.
Click to expand...

And I also have some doubts if it is really usefull, because stopping a VM is not really reliable (windows is kniown to have problems)
 
dietmar said:
...windows is kniown to have problems
Click to expand...

I also have had, but I have completely solved it. I've watched backup logs for a while (it's being sent me on email) therefore I'm always convinced that it works as good as needed.

Now all of my machines are backing up using "snapshot" mode on Tuesday and by "stop mode" on Saturday. Because snapshot mode insufficiently reliable while a guest machine uses write cache or runs any kind of database engine.
 
Last edited:
dietmar said:
And I also have some doubts if it is really usefull, because stopping a VM is not really reliable (windows is kniown to have problems)
Click to expand...
I'd like to set eyes on this problem again.
When I do snapshot backup up without stopping machine I have so called "crash consistent backup" when files system is consistent but some applications probably didn't save all their data.
For example: mail server which is working on virtual machine have received email message and is going to proccess it. It can be spam filter or post-convertion or extracting attaches or some else. Backup can occur while proccess haven't yet done and some data will be lost. In another case when I stop machine all services are going to be stoped to and no data will be lost. It's true for databases and many other applications too.
 
I do this by hand:
- stop kvm with windows
- make snapshot
- start windows kvm
- attach snapshot to a kvm with linux
- start linux kvm and run ntfsclone
- stop linux kvm
- remove snapshot
I think images are smaller that way and backup is reliable. I wish there would be an automated process for this.
 
vlad said:
I do this by hand:
- stop kvm with windows
- make snapshot
- start windows kvm
- attach snapshot to a kvm with linux
- start linux kvm and run ntfsclone
- stop linux kvm
- remove snapshot
I think images are smaller that way and backup is reliable. I wish there would be an automated process for this.
Click to expand...
Could you explain what the reason to use another linux for running ntfsclone? Looks like it can be produced on the main host.
 
giner said:
Could you explain what the reason to use another linux for running ntfsclone? Looks like it can be produced on the main host.
Click to expand...

Internal partitions of snapshot are not visible on the host unless you install multipath tools or ntfsclone wont work on raw drive. I dont want to sabotage hosts reliability by installing multipath tools, in past those would create strange problems where I could not unmount/remove snapshots.
 
vlad said:
Internal partitions of snapshot are not visible on the host unless you install multipath tools or ntfsclone wont work on raw drive. I dont want to sabotage hosts reliability by installing multipath tools, in past those would create strange problems where I could not unmount/remove snapshots.
Click to expand...
Sorry, I don't quite see what you mean. I can do snapshot and then just copy image file (raw or qcow2) to another place.
 
giner said:
Sorry, I don't quite see what you mean. I can do snapshot and then just copy image file (raw or qcow2) to another place.
Click to expand...

I don't use image files because those should be slowing down guest machines, I use LVMs instead, for optimal speed.
 
vlad said:
I don't use image files because those should be slowing down guest machines, I use LVMs instead, for optimal speed.
Click to expand...
Of course. I mean you can stop VM, create LVM snapshot, start VM, mount snapshot to some directory, copy VM images to backup place, unmount and remove snapshot.
 
giner said:
Of course. I mean you can stop VM, create LVM snapshot, start VM, mount snapshot to some directory, copy VM images to backup place, unmount and remove snapshot.
Click to expand...

In my case, you can't do that. There are no VM images(no raw and no qcow), KVMs use whole LVM block devices, i.e. /dev/pve/vm-102-disk-1
 
vlad said:
In my case, you can't do that. There are no VM images(no raw and no qcow), KVMs use whole LVM block devices, i.e. /dev/pve/vm-102-disk-1
Click to expand...
Code: 
lvcreate --size 1G --snapshot --name snap /dev/pve/vm-102-disk-1
dd if=/dev/pve/snap of=/mnt/backup/vm-102-disk-1-image bs=1M
lvremove /dev/pve/vm-102-disk-1
Can you do this way?
 
giner said:
Code: 
lvcreate --size 1G --snapshot --name snap /dev/pve/vm-102-disk-1
dd if=/dev/pve/snap of=/mnt/backup/vm-102-disk-1-image bs=1M
lvremove /dev/pve/vm-102-disk-1
Can you do this way?
Click to expand...

Yes, but its a waste of backup space if there is not much data. What I need to do is:
ntfsclone -s -o backupfile.ntfs /dev/pve/snap0p1
or
ntfsclone -s -o backupfile.ntfs /dev/pve/snap0p5
where snap0p1 primary partition in KVMs drive and snap0p5 is a secondary.

if vm-102-disk-1 is 100gb, and snap0p1 uses only 15gb, snap0p5 using only 25gb then my backups will be only 40gb in size and your backups will be whole 100gb, do you understand now?
 
You must log in or register to reply here.
Top Bottom