vzdump : bwlimit modified but no speed increase...

X

XZed

Member
Sep 20, 2009
74
0
6
vzdump : bwlimit modified but no speed increase (or decrease)...

Edit :

I tried today to decrease bwlimit, but speed stay at the same value : 7/8 Mo/s....

222: Nov 18 08:29:01 INFO: Starting Backup of VM 222 (qemu)
222: Nov 18 08:29:01 INFO: running
222: Nov 18 08:29:01 INFO: status = running
222: Nov 18 08:29:02 INFO: backup mode: snapshot
222: Nov 18 08:29:02 INFO: bandwidth limit: 5120 KB/s
222: Nov 18 08:29:02 INFO: Logical volume "vzsnap-virtual-0" created
222: Nov 18 08:29:02 INFO: creating archive '/mnt/backup/vzdump-qemu-222-2009_11_18-08_29_01.tgz'
222: Nov 18 08:29:02 INFO: adding '/mnt/backup/vzdump-qemu-222-2009_11_18-08_29_01.tmp/qemu-server.conf' to archive ('qemu-server.conf')
222: Nov 18 08:29:02 INFO: adding '/mnt/vzsnap0/images/222/vm-222-disk-1.raw' to archive ('vm-disk-ide0.raw')
222: Nov 18 08:36:59 INFO: Total bytes written: 3759320064 (7.52 MiB/s)
222: Nov 18 08:36:59 INFO: archive file size: 2.32GB
222: Nov 18 08:36:59 INFO: delete old backup '/mnt/backup/vzdump-qemu-222-2009_11_17-18_17_01.tgz'
222: Nov 18 08:37:02 INFO: Logical volume "vzsnap-virtual-0" successfully removed
222: Nov 18 08:37:02 INFO: Finished Backup of VM 222 (00:08:01)

------------------------------------------------------------------------------------------------------------------------------

Hello,

In a previous post i was talking about older vzdump versions :

http://www.proxmox.com/forum/showthread.php?p=14254#post14254

Dietmar supposed it was a bug (fortunately, it was a benefit for me :p)...

But now, unfortunately, i have a bug, but it isn't benefit for me lol :eek:...

Since v1.4, indeed, the backups really occur at 10 Mo/s.

So i modified bwlimit value to "102400" and rebooted...

But, in a local dir (/mnt/backup), despite the value has changed, i don't bypass 10 Mo/s limit...
222: Nov 17 18:17:01 INFO: Starting Backup of VM 222 (qemu)
222: Nov 17 18:17:01 INFO: running
222: Nov 17 18:17:01 INFO: status = running
222: Nov 17 18:17:02 INFO: backup mode: snapshot
222: Nov 17 18:17:02 INFO: bandwidth limit: 102400 KB/s
222: Nov 17 18:17:02 INFO: Logical volume "vzsnap-virtual-0" created
222: Nov 17 18:17:02 INFO: creating archive '/mnt/backup/vzdump-qemu-222-2009_11_17-18_17_01.tgz'
222: Nov 17 18:17:02 INFO: adding '/mnt/backup/vzdump-qemu-222-2009_11_17-18_17_01.tmp/qemu-server.conf' to archive ('qemu-server.conf')
222: Nov 17 18:17:02 INFO: adding '/mnt/vzsnap0/images/222/vm-222-disk-1.raw' to archive ('vm-disk-ide0.raw')
222: Nov 17 18:24:25 INFO: Total bytes written: 3759308800 (8.09 MiB/s)
222: Nov 17 18:24:25 INFO: archive file size: 2.32GB
222: Nov 17 18:24:26 INFO: Logical volume "vzsnap-virtual-0" successfully removed
222: Nov 17 18:24:26 INFO: Finished Backup of VM 222 (00:07:25)
I thought it was a misunderstood about Mbs and Mo/s, but it isn't...

Tomorrow i will make further tests...

If i need to give you more details, tell me it...

Thanks,

Sincerely,
 
Last edited:
dietmar said:
What kind of storage is mounted at /mnt/backup - how fast is it?
Click to expand...

Well, i created /mnt/backup but no external storage is mounted, so it's local, but to simplify, i creatd a new local backup dir for no more confusion : /backup.

An i modified bwlimit to 1 Gbs to haven't more doubts lol !

And i can confirm that my local RAID has greater speed than 8 Mo/s ^^...


222: Nov 18 09:19:02 INFO: Starting Backup of VM 222 (qemu)
222: Nov 18 09:19:02 INFO: running
222: Nov 18 09:19:02 INFO: status = running
222: Nov 18 09:19:02 INFO: backup mode: snapshot
222: Nov 18 09:19:02 INFO: bandwidth limit: 1024000 KB/s
222: Nov 18 09:19:02 INFO: Logical volume "vzsnap-virtual-0" created
222: Nov 18 09:19:02 INFO: creating archive '/backup/vzdump-qemu-222-2009_11_18-09_19_02.tgz'
222: Nov 18 09:19:02 INFO: adding '/backup/vzdump-qemu-222-2009_11_18-09_19_02.tmp/qemu-server.conf' to archive ('qemu-server.conf')
222: Nov 18 09:19:02 INFO: adding '/mnt/vzsnap0/images/222/vm-222-disk-1.raw' to archive ('vm-disk-ide0.raw')
222: Nov 18 09:26:13 INFO: Total bytes written: 3759320064 (8.32 MiB/s)
222: Nov 18 09:26:13 INFO: archive file size: 2.32GB
222: Nov 18 09:26:13 INFO: delete old backup '/backup/vzdump-qemu-222-2009_11_18-09_08_02.tgz'
222: Nov 18 09:26:16 INFO: Logical volume "vzsnap-virtual-0" successfully removed
222: Nov 18 09:26:16 INFO: Finished Backup of VM 222 (00:07:14)


Thank you,

Sincerely,
 
XZed said:
And i can confirm that my local RAID has greater speed than 8 Mo/s ^^...
Click to expand...

Sure, but the disk is used for many things (VMs, snapshot, vzdump, ...)

Do you have a high IO load or high CPU load during backup?
 
dietmar said:
So the vzdump process uses 100% CPU (use ps or top to verify)?
Click to expand...

Oh right... i haven't noticed that 7% cpu usage during backup approximately corresponded to 1 core / 16 cores.

And, as you supposed, during backup, i used top command and noticed that "gzip" process was using all cpu resource (sometimes a small 1% cpu usage appeared used by "vmtar" process).

Tomorrow i will try the same backups but without compression...

But what makes me think that it isn't related to gzip cpu usage is that :

even if i specify a low bwlimit, it bypasses this limit staying around the same value (~ 8Mo/s)...

The real problem seems to be that backup task doesn't take car about bwlimit value...
 
Without compression, only vmtar takes few cpu usage (~ 5%) and, indeed, write speed increased but still is, imho, very little for a local backup (i compared with other setups that has previous proxmox and slower servers : these ones backup at 45/50 Mo/s on a local dir)...


222: Nov 19 09:01:02 INFO: Starting Backup of VM 222 (qemu)
222: Nov 19 09:01:02 INFO: running
222: Nov 19 09:01:02 INFO: status = running
222: Nov 19 09:01:02 INFO: backup mode: snapshot
222: Nov 19 09:01:02 INFO: bandwidth limit: 1024000 KB/s
222: Nov 19 09:01:02 INFO: Logical volume "vzsnap-virtual-0" created
222: Nov 19 09:01:02 INFO: creating archive '/backup/vzdump-qemu-222-2009_11_19-09_01_02.tar'
222: Nov 19 09:01:02 INFO: adding '/backup/vzdump-qemu-222-2009_11_19-09_01_02.tmp/qemu-server.conf' to archive ('qemu-server.conf')
222: Nov 19 09:01:02 INFO: adding '/mnt/vzsnap0/images/222/vm-222-disk-1.raw' to archive ('vm-disk-ide0.raw')
222: Nov 19 09:05:10 INFO: Total bytes written: 3759407616 (14.46 MiB/s)
222: Nov 19 09:05:10 INFO: archive file size: 3.50GB
222: Nov 19 09:05:10 INFO: delete old backup '/backup/vzdump-qemu-222-2009_11_19-08_55_01.tar'
222: Nov 19 09:05:16 INFO: Logical volume "vzsnap-virtual-0" successfully removed
222: Nov 19 09:05:16 INFO: Finished Backup of VM 222 (00:04:14)
 
dietmar said:
And how large is 'vm-disk-ide0.raw'?
Click to expand...

same as vm-222-disk-1.raw "untarred" (40 Go):

-rw-r--r-- 1 root root 42949672960 Nov 19 09:01 vm-disk-ide0.raw

But as it's sparse file , it only "really use" (Cf. "df -h") less than 4 Go...

Proof ? The tar size (so not compressed but only archived) :

-rw-r--r-- 1 root root 3759407616 Nov 19 09:05 vzdump-qemu-222-2009_11_19-09_01_02.tar

OK, will make further tests...

Thanks again
 
I tried in "stop" mode, it's quite the same speed..

222: Nov 19 15:28:02 INFO: Starting Backup of VM 222 (qemu)
222: Nov 19 15:28:02 INFO: running
222: Nov 19 15:28:02 INFO: status = running
222: Nov 19 15:28:02 INFO: backup mode: stop
222: Nov 19 15:28:02 INFO: bandwidth limit: 1024000 KB/s
222: Nov 19 15:28:02 INFO: stopping vm
222: Nov 19 15:28:16 INFO: creating archive '/backup/vzdump-qemu-222-2009_11_19-15_28_02.tar'
222: Nov 19 15:28:16 INFO: adding '/backup/vzdump-qemu-222-2009_11_19-15_28_02.tmp/qemu-server.conf' to archive ('qemu-server.conf')
222: Nov 19 15:28:16 INFO: adding '/var/lib/vz/images/222/vm-222-disk-1.raw' to archive ('vm-disk-ide0.raw')
222: Nov 19 15:32:10 INFO: Total bytes written: 3759367680 (15.32 MiB/s)
222: Nov 19 15:32:10 INFO: archive file size: 3.50GB
222: Nov 19 15:32:10 INFO: delete old backup '/backup/vzdump-qemu-222-2009_11_19-09_01_02.tar'
222: Nov 19 15:32:16 INFO: restarting vm
222: Nov 19 15:32:16 INFO: vm is online again after 254 seconds
222: Nov 19 15:32:16 INFO: Finished Backup of VM 222 (00:04:14)

I'm trying further tests...

Thanks
 
XZed said:
Well, i created /mnt/backup but no external storage is mounted, so it's local, but to simplify, i creatd a new local backup dir for no more confusion : /backup.

An i modified bwlimit to 1 Gbs to haven't more doubts lol !

And i can confirm that my local RAID has greater speed than 8 Mo/s ^^...
Click to expand...

If the same RAID array is doing the reading and writing to itself at the same time (not to mention snapshot making, which was never fast), I would not expect it to be any faster than what you get: a couple of megabytes per second. In fact, even that speed is only possible because of read-aheads and efficient caching on the controller.

To effectively gauge the speed of vzdump, mount a fast, empty disk (or gigabit network share) as /backup, and create your snapshot backups there. Try it with several bwlimit speed settings from 5 to 20 megabytes per sec, as a modern single disk can cope with such speeds easily. Post your findings.
 
Last edited:
XZed said:
same as vm-222-disk-1.raw "untarred" (40 Go):

-rw-r--r-- 1 root root 42949672960 Nov 19 09:01 vm-disk-ide0.raw

But as it's sparse file
Click to expand...

To generate a sparse archive, we have to read the file twice (due to limitations of the tar archive format). So we have to read 80GB in 4 minutes ==> 330MB/s - i guess your disk is not faster.
 
hmmmm... well, i couldn't make further tests, but i'll try the same vm on a qcow2 disk and will make feedback...
 
gkovacs said:
If the same RAID array is doing the reading and writing to itself at the same time (not to mention snapshot making, which was never fast), I would not expect it to be any faster than what you get: a couple of megabytes per second. In fact, even that speed is only possible because of read-aheads and efficient caching on the controller.

To effectively gauge the speed of vzdump, mount a fast, empty disk (or gigabit network share) as /backup, and create your snapshot backups there. Try it with several bwlimit speed settings from 5 to 20 megabytes per sec, as a modern single disk can cope with such speeds easily. Post your findings.
Click to expand...

As i previously answered to dietmar :

Indeed, i'm going to setup various configurations of this same vm and tell you the results. Thanks again for support.
 
Well, after many tests the conclusion is the same :

VM w/ RAW format are backed up very slowly compared to VM w/ QCOW2 format....

Different VM tried...

And i had two clones : one raw format, other qcow2 format...

With raw format : 6 Mo/s backup

With qcow2 : quite 50 Mo/s backup....

Is it normal or a bug ?

Thank you very much,

Sincerely,
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back