scp speed (MB/s) between VMs on same host

[tuathan]

I am testing some file transfer speeds in PVE 6.2-4.

I have got two VMs on a host both using same bridge (with no ports/slave) for private network (192.168.1.X).

They are both using VIRTIO network device, a zfs pool for harddisk space and have 3GB or RAM and 1 socket, 2 Core CPU.

The file transfer speed for a 100 MB file is ~85 - 90 MB / s. What is limiting this speed?

(This PVE host is part of a cluster and I am getting > 100 MB / s scp speed between physical host nodes)

Thanks.

 

[aaron]

Are they using the same storage? Reading and writing to the same storage might be the limiting factor. What bus/device type do the disks of the guests use?
Another one could be the nodes CPU as the sending / receiving and network is happening purely on that one node.

 

[tuathan]

Are they using the same storage? Reading and writing to the same storage might be the limiting factor. What bus/device type do the disks of the guests use?
Another one could be the nodes CPU as the sending / receiving and network is happening purely on that one node.

Yes the VMs share the same ZFS (2 mirrored SAS disks) pool on a Dell R630 server (hba mode), it most likely is this I suppose.
The server has 2 x 12 Cores so I would doubt it's CPU?

In terms of VMs on a private network (bridge) with no ports/slave I assume sending data via scp on the private network is effectively just reading/writing to different parts of the zfs pool.