To track that down, simply test read speed of those volumes on the NAS (using dd or cat).
2 semi-identical VMs, different back up speed
- Thread starter wahmed
- Start date
To track that down, simply test read speed of those volumes on the NAS (using dd or cat).
I ran dd on the NAS volume where backup gets dumped.
Write
=====
$> dd if=/dev/zero of=test.img bs=8k count=256k
262144+0 records in
262144+0 records out
2048 MB transferred in 5.07 secs (404 MB/sec)
Read
====
$> dd if=test.img of=/dev/null of=test.img bs=8k
262144+0 records in
262144+0 records out
2048 MB transferred in 18.27 secs (112 MB/sec)
Whery strange readings you have.
My readings:
$ sudo dd if=/dev/zero of=test.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 8.27238 s, 260 MB/s
$ sudo dd if=test.img of=/dev/null bs=8k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 11.2562 s, 191 MB/s
My readings:
$ sudo dd if=/dev/zero of=test.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 8.27238 s, 260 MB/s
$ sudo dd if=test.img of=/dev/null bs=8k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 11.2562 s, 191 MB/s
Opps, so sorry. Here is the dd again. This time i ran it inside the VM1 and VM2. I hope this is what you meant.
VM1 Write
========
$> dd if=/dev/zero of=test.img bs=8k count=256k
2.1 GB transferred in 47.52 secs (45.25 MB/sec)
VM1 Read
=======
$> dd if=test.img of=/dev/null of=test.img bs=8k
2.1GB transferred in 30.94 secs (69.4 MB/sec)
VM2 Write
========
$> dd if=/dev/zero of=test.img bs=8k count=256k
2.1 GB transferred in 69.49 secs (30.9 MB/sec)
VM2 Read
=======
$> dd if=test.img of=/dev/null of=test.img bs=8k
2.1GB transferred in 29.25 secs (73.4 MB/sec)
Not really, but this test also shows that speeds are different, although disks are one the same NAS.
I wonder.
1) Does the two vm's use the same kind of file system?
2) Does the two vm's have access to the same network hardware?
3) Does the two vm's run on the same node or are the running on two different nodes?
4) In case they run on different nodes does the nodes have the same hardware specs?
1) Does the two vm's use the same kind of file system?
2) Does the two vm's have access to the same network hardware?
3) Does the two vm's run on the same node or are the running on two different nodes?
4) In case they run on different nodes does the nodes have the same hardware specs?
A miracle has happened!!! My back up speed has increased significantly. So far i did several backups and the result has been consistent. All i did was take create a NFS share on the shared FreeNAS storage, convert all raw VM hdds to qcow2, restore all VM on to NFS share from iSCSI, delete iSCSI. Followig are the before and after result:
Before changes, all VMs on iSCSI share. All hdds are in RAW format:
VM 1
====
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 2 hr 38 mins
VM 2
====
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 35 mins
After changes, all VMs on NFS share. All hdds are in QCOW2 format:
VM 1
====
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 37 mins
VM 2
====
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 2 mins
Does it make sense to everybody. Keep in mind, no hardware changes has been made.
Before changes, all VMs on iSCSI share. All hdds are in RAW format:
VM 1
====
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 2 hr 38 mins
VM 2
====
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 35 mins
After changes, all VMs on NFS share. All hdds are in QCOW2 format:
VM 1
====
Allocated HDD = 400GB (virtio)
Used Space = 55GB
Backup Time = 37 mins
VM 2
====
Allocated HDD = 200GB
Used Space = 115GB
Backup Time = 1 hr 2 mins
Does it make sense to everybody. Keep in mind, no hardware changes has been made.
I see you use FreeNAS so therefore I am not surprised. The iSCSI implementation in FreeBSD is a joke and the performance is so bad that I render it broken. Using the iSCSI implementation in Linux is at least double as fast and the Comstar iSCSI implementation in Solaris and derivatives are 4 - 5 times faster. Since iSCSI in general are faster and less error prone than NFS I would never recommend anybody using FreeBSD based NAS as a storage solution for anything vital.
Mir, I cannot thank you enough for pointing me to OmniOS direction. I have successfully deployed an OmniOS Server with Napp-it and the transfer speed has jumped like i never seen before. OmniOS trying to use the entire gigabit speed which cut down back up time significantly. Solaris Linux is completely new domain for me. I have lived with RPM based linux for very very long time. Feels like i am relearning linux all over again.
With Proxmox and OmniOS, the network is much better shape than it had been. I have not done much tweaking yet. Still getting used the interface and commands. Hopefully future tweaking will bring significant performance. FreeNAS worked out pretty good but the speed was just not acceptable. Anybody looking for faster data transfer rate give OmniOS and Napp-it a try.