Poor network performance with Zeroshell VM

[tommisan]

Hi Everyone,

I am using a OpenVPN server (Zeroshell 3.8.2, Linux distribution) on a Proxmox VM environment (enterprise repo).

The problem is that the vm has really poor network performance on a full Gb network.

Best (unexpected) results are with Intel e1000 and vmxnet3 driver, but we are talking about max 6 MB/s on Gb NIC. VirtIO performs around 1MB/s.

I tested on a Proxmox cluster (4.4) and on a single node (5.1), similar results.

With a similar Zeroshell setup on a very old physical server I can easily get 24 MB/s.

No firewall on Proxmox, NIC are in Linux bridge mode.

I would appreciate any ideas.
Thanks

 

[LnxBil]

Please post all VM settings.

 

[tommisan]

This is VM config with virtio driver (which I generally use in linux vm):

bootdisk: scsi0
cores: 1
ide2: none,media=cdrom
memory: 4092
name: zeroshell
net0: virtio=...,bridge=vmbr0
net1: virtio=..,bridge=vmbr1
net2: virtio=...,bridge=vmbr2
numa: 0
ostype: l26
scsi0: qnap:103/vm-103-disk-1.qcow2,size=8G
scsihw: virtio-scsi-pci
smbios1: uuid=...
sockets: 1

 

[LnxBil]

Best (unexpected) results are with Intel e1000 and vmxnet3 driver, but we are talking about max 6 MB/s on Gb NIC. VirtIO performs around 1MB/s.

Is this the encrypted performance or the unencrypted?

 

[tommisan]

This is the speed of a file transfert from a physical server to my pc through VPN (zeroshell vm) - encrypted performance.

 

[LnxBil]

What is the performance of without encryption? Could you please perform a speedtest of openssl inside of your VM:

https://lwn.net/Articles/269327/

Please also check what encryption you're using and retest that specific cipher with openssl to get some "real" numbers.

 

[Symbol]

I am using a OpenVPN server (Zeroshell 3.8.2, Linux distribution) on a Proxmox VM environment (enterprise repo).
The problem is that the vm has really poor network performance on a full Gb network.

For encrypted traffic? If so, it could help to expose AES CPU Flag to the VM.
So, try to set the VM CPU type to "host", shutdown the VM, then start it again.

 

[tommisan]

log vpn

2018-01-24 16:44:39 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
2018-01-24 16:44:39 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
2018-01-24 16:44:39 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA

test openssl speed

The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
bf-cbc 73126.65k 80542.50k 83113.21k 84060.81k 84526.05k

The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 46269.65k 51255.30k 52734.89k 83771.73k 84243.40k

 

[tommisan]

test with different CPU core and driver:

VMXNET3 driver
2 core (CPU set to default kvm64)-> 10 MB/s (CPU 70% under file transfert, default use CPU 5%)

E1000 driver
1 core (CPU set to default kvm64) -> 6 MB/s (CPU about 130% under file transfert, default use CPU 7%)
4 core (CPU set to default kvm64)-> 9.2 MB/s (CPU 40% under file transfert, default use < CPU 3%)
4 core (CPU set to host) -> 9.5 MB/s (CPU 40% under file transfert, default use < CPU 3%)

virtio driver
1 core (CPU set to default kvm64) -> about 1 MB/s (CPU about 20% under file transfert, default use CPU 7%)
4 core (CPU set to default kvm64)-> about 1 MB/s (CPU 7% under file transfert, default use < CPU 3%)
4 core (CPU set to host)-> about 1 MB/s (CPU 5% under file transfert, default use < CPU 3%)

RTL8139 driver
2 core (CPU set to default kvm64)-> about 9 MB/s (CPU 80% under file transfert)

 

[LnxBil]

That's really strange. The benchmarks look good, so it is not an encryption throughput problem. Maybe some offloading does not work correctly inside of the guest?

Are your benchmarks done with or without encryption? If they were done with encryption, please benchmark without to get the base line.

 

[tommisan]

I did some test with iperf3
It seems a problem with the VPN logical/software interface (maybe related to openvpn?)

On the public physical NIC (ethx) I get 65 MB/s, which is reasonable value. (In the private NIC I get 90 MB/s)

On the VPN logical NIC (VPN99) I get the same 10-12 MB/s

Could be also interesting to know the max speed of a OpenVPN server (TCP, encrypted) on a Gb NIC..I got 24 MB/s on an old server

 

[LnxBil]

Could be also interesting to know the max speed of a OpenVPN server (TCP, encrypted) on a Gb NIC..I got 24 MB/s on an old server

Depends heavily on the used encryption algorithm, and you normally get better throughput with UDP.

 

[tommisan]

Latest tests:
openvpn with tcp and udp showed similar values (8-11MB/s). udp performed better but still close to tcp values
openvpn without encrpytion (--cypher none): 12.5 MB/s

In another datacenter with a openvpn server (tcp, encrypted) installed on a brand new physical server I got 24 MB/s (Gb network).

Has anyone achieved better results of 24 MB/s in a Gb network with openvpn?