2 semi-identical VMs, different back up speed

[wahmed]

Hello,
I got a 2 Linux VM Server on the same node with following setup.
VM 1
====
CPU = 1
RAM = 4gb
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 2 hr 38 mins

VM 2
====
CPU = 1
RAM = 1GB
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 35 mins

My question is, even though VM 2 has more data to back up, it actually finishes full back up faster than VM 1, why is that? Is it because the allocated virtual drive is larger in the VM 1?

 

[dietmar]

Is it because the allocated virtual drive is larger in the VM 1?

Yes. How big are the final backup files?

 

[wahmed]

VM1 final backup = 51.3gb
VM2 final backup = 115.3gb

 

[dietmar]

With or without compression (also test without compression).

 

[wahmed]

VM1
-----
Thick provisioned HDD = 400GB RAW
With Compression Backup time = 2 hr 18 mins
With Compression Backup Size = 53.62GB
Without Compression Backup time = 4 hr 39 mins
Without Compression Backup Size = 63.23GB

VM2
-----
Thick provisioned HDD = 200GB RAW
With Compression Backup time = 1 hr 40 mins
With Compression Backup Size = 117.68GB
Without Compression Backup time = 3 hr 48 mins
Without Compression Backup Size = 120.46GB

 

[dietmar]

Seems that read speed is the limiting factor.

 

[wahmed]

Even though both VMs on same NAS they are having 2 different read speed? Any way to increase that from Proxmox?

 

[mir]

just a guess: Are the cables of equal length and quality (CAT 5e, CAT 6 etc)

 

[wahmed]

just a guess: Are the cables of equal length and quality (CAT 5e, CAT 6 etc)

Yep, all nodes are connected to a switch with 6' cat5e cables.

 

[mir]

Then I guess your switch is awaiting to be replaced by some of higher quality.

 

[wahmed]

Then I guess your switch is awaiting to be replaced by some of higher quality.

The switch is 3 months old Netgear 24 Port Gigabit Smart Switch GS724T. To make sure i dont have bad ports, i even tried to switch the ports. The Proxmox node that trying to do all the back up is i7-3770 with 32GB Ram. All nodes and the NAS has identical Intel Pro 10/1000 NIC.

 

[mir]

I that case my guess is a bad cable. Have you tried switching the cables to verify equal speed when using different cables.

 

[wahmed]

I that case my guess is a bad cable. Have you tried switching the cables to verify equal speed when using different cables.

All the cables were replaced ready made Cat 5e network cable 3 months ago along with the new switch. I also switched node cables just to make sure i did not end up with faulty cables. Result is same.

 

[dietmar]

Even though both VMs on same NAS they are having 2 different read speed?

That usually depends where data is physically located on the disks (inner or outer cylinders), and fragmentation.

Or do you backup while the VMs are running?

 

[wahmed]

Or do you backup while the VMs are running?

That is a good point. I run both VM backup on Snapshot mode, never shutdown or stop. I will run some test with VMs completely shut down and see how it goes.

 

[dietmar]

That is a good point. I run both VM backup on Snapshot mode

This can seriously affect performance.

- - - Updated - - -

That is a good point.

The other point is also good ;-) Read speed can easily differ by factor 2 depending on physical sector location.

 

[wahmed]

The other point is also good ;-) Read speed can easily differ by factor 2 depending on physical sector location.

My shared storage for VM is on FreeNAS with ZFS in Software RAID and 4x2TB HDD. Isnt it little too difficult to determine the read speed of all 4 HDD and their physical sectors allocation?

 

[wahmed]

Just did another backup of VM1 with the vm completely shutdown. Result is confusing.

Backup time in Snapshot mode: 2 hr 12 mins
Backup time with shutdown : 2 hr 1 min.
Any thoughts?

 

[dietmar]

Backup time in Snapshot mode: 2 hr 12 mins
Backup time with shutdown : 2 hr 1 min.

Why is that confusing? - time is almost equal, which is expected.

 

[wahmed]

It was confusing because i always thought running VM would take more time to backup than a shutdown VM. Last night i did another usual backup. This time vm2 with 120GB of data got backed up in 1 hr 10 mins on the same NAS and the vm1 with 57GB of data took 2 hr 20 mins. The only thing is i backed up vm1 with 57GB on a 2nd backup NAS which has identical setup as NAS1. This 2nd NAS is empty. So even both backups were done on 2 separate physical machine, results were same.

Its not causing any issue. Backups are being done just fine. Just a mystery.