2 semi-identical VMs, different back up speed

To track that down, simply test read speed of those volumes on the NAS (using dd or cat).

I ran dd on the NAS volume where backup gets dumped.
Write
=====
$> dd if=/dev/zero of=test.img bs=8k count=256k
262144+0 records in
262144+0 records out
2048 MB transferred in 5.07 secs (404 MB/sec)

Read
====
$> dd if=test.img of=/dev/null of=test.img bs=8k
262144+0 records in
262144+0 records out
2048 MB transferred in 18.27 secs (112 MB/sec)
 
I have one VM - 10GB that backs up at 350mb/s. I have a 100gb that backs up at 50mb/s - its normal. However, disk format also affects
 
Whery strange readings you have.

My readings:
$ sudo dd if=/dev/zero of=test.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 8.27238 s, 260 MB/s
$ sudo dd if=test.img of=/dev/null bs=8k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 11.2562 s, 191 MB/s
 
The suggestion was to test read speed to the real VM images.

Opps, so sorry. Here is the dd again. This time i ran it inside the VM1 and VM2. I hope this is what you meant.

VM1 Write
========
$> dd if=/dev/zero of=test.img bs=8k count=256k
2.1 GB transferred in 47.52 secs (45.25 MB/sec)

VM1 Read
=======
$> dd if=test.img of=/dev/null of=test.img bs=8k
2.1GB transferred in 30.94 secs (69.4 MB/sec)

VM2 Write
========
$> dd if=/dev/zero of=test.img bs=8k count=256k
2.1 GB transferred in 69.49 secs (30.9 MB/sec)

VM2 Read
=======
$> dd if=test.img of=/dev/null of=test.img bs=8k
2.1GB transferred in 29.25 secs (73.4 MB/sec)
 
I wonder.
1) Does the two vm's use the same kind of file system?
2) Does the two vm's have access to the same network hardware?
3) Does the two vm's run on the same node or are the running on two different nodes?
4) In case they run on different nodes does the nodes have the same hardware specs?
 
A miracle has happened!!! My back up speed has increased significantly. So far i did several backups and the result has been consistent. All i did was take create a NFS share on the shared FreeNAS storage, convert all raw VM hdds to qcow2, restore all VM on to NFS share from iSCSI, delete iSCSI. Followig are the before and after result:

Before changes, all VMs on iSCSI share. All hdds are in RAW format:
VM 1
====
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 2 hr 38 mins

VM 2
====
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 35 mins

After changes, all VMs on NFS share. All hdds are in QCOW2 format:
VM 1
====
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 37 mins

VM 2
====
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 2 mins

Does it make sense to everybody. Keep in mind, no hardware changes has been made.
 
I see you use FreeNAS so therefore I am not surprised. The iSCSI implementation in FreeBSD is a joke and the performance is so bad that I render it broken. Using the iSCSI implementation in Linux is at least double as fast and the Comstar iSCSI implementation in Solaris and derivatives are 4 - 5 times faster. Since iSCSI in general are faster and less error prone than NFS I would never recommend anybody using FreeBSD based NAS as a storage solution for anything vital.
 
Whery strange readings you have.

My readings:
$ sudo dd if=/dev/zero of=test.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 8.27238 s, 260 MB/s
$ sudo dd if=test.img of=/dev/null bs=8k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 11.2562 s, 191 MB/s

What do you use for shared storage? Your transfers rates are stunning compare to mine.
 
I use Omnios as shared storage With napp-it as web baser GUI.

Mir, I cannot thank you enough for pointing me to OmniOS direction. I have successfully deployed an OmniOS Server with Napp-it and the transfer speed has jumped like i never seen before. OmniOS trying to use the entire gigabit speed which cut down back up time significantly. Solaris Linux is completely new domain for me. I have lived with RPM based linux for very very long time. Feels like i am relearning linux all over again. :) With Proxmox and OmniOS, the network is much better shape than it had been. I have not done much tweaking yet. Still getting used the interface and commands. Hopefully future tweaking will bring significant performance.
FreeNAS worked out pretty good but the speed was just not acceptable. Anybody looking for faster data transfer rate give OmniOS and Napp-it a try.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!