ESXi to PVE: qm importdisk speed

[luca666]

Hi All.

Astonished of that:

Scenario1
ESXi+1gbe and PVE+1gbe
mapped "my221" as ESXi storage
qm importdisk 119 /run/pve/import/esxi/my221/mnt/ha-datacenter/ssd1/srvx2/srvx2.vmdk local-zfs --format raw
speed: 150gb/hour
ESXi network performance shows 1gbit bandwith usage
PVE network performance shows 1gbit bandwith usage

Scenario2
ESXi+40gbe and PVE+40gbe
mapped "my221" as ESXi storage, obviously via 40gbe nic
qm importdisk 119 /run/pve/import/esxi/my221/mnt/ha-datacenter/ssd1/srvx2/srvx2.vmdk local-zfs --format raw
speed: 150gb/hour (???)
ESXi network performance shows 40gbit bandwith usage
PVE network performance shows 40gbit bandwith usage

Any idea?
(both are hw ok, VMs not running, iperf3 ok, ...)

Thanks
L.

 

[bfrd9k]

Tell me more about source and destination storage, have you banchmarked them? How about MTU's on all interfaces and switcheports?

Also, can you do me a favor and use standard transfer speeds, network in mbps (megabit per second), or gbps (gigabit per second), or storage in MB/s (megabyte per second), GB/s.

 

[luca666]

Tell me more about source and destination storage, have you banchmarked them? How about MTU's on all interfaces and switcheports?

Also, can you do me a favor and use standard transfer speeds, network in mbps (megabit per second), or gbps (gigabit per second), or storage in MB/s (megabyte per second), GB/s.

Thanks for your reply.

Source and destination HW:
HPE DL360gen9
2x Xeon E5-2690v4 2.60GHz
192gb ram
P440AR controller + 2x ssd kingston SEDC500M/3840G, raid1

Source: esxi 5.5, VMFS-5
Destination: PVE 8.3.1, zfs

Nic speeds are 1gbps and 40gbps.
Data transfer is 150GB/hour = 41MB/s.
MTU is 1500 on all nics/switchports.

The very weird thing is that yesterday (without 40gbe nics) I tried a migration obtaining a 150GB/hour speed, about the half of the 1gbps capabilities, but ok for me...

This morning I installed and configured two 40gbe nics and tried again.

ESXi chart (outgoing traffic):


PVE chart (incoming traffic):


Both are measuring 40gbps but after a while I stopped everything realizing the real data trasfer is still 150GB/hour.
Philosophical question : what are they talking about at 40gbps if result is 150GB/hour (about 400mbps)???

Thanks
L.

 

[bfrd9k]

The ESXi graph is more descriptive, 45000KB/s is about 360mbps (45MB/s).

Can you benchmark reads from the source and writes to the destination? My guess is that your bottleneck will be storage writes but it should be at least 200MB/s sustained sequential writes.

Use a tool like fio.