looks interesting. I just checked if I got such a CPU in our testlab (http://ark.intel.com/search/advanced/?s=t&AESTech=true) but it looks not good so I cannot test this here.
Speed up the VM (storage) migration
- Thread starter xlnt
- Start date
similar results with X5650:
vcluster1a:~# time scp -c blowfish /var/lib/vz/template/cache/centos-6.0-standard_6.0-1_amd64.tar.gz vcluster1b.int:/tmp
centos-6.0-standard_6.0-1_amd64.tar.gz 100% 203MB 40.6MB/s 00:05
real 0m5.910s
user 0m2.852s
sys 0m0.556s
vcluster1a:~# time scp -c arcfour /var/lib/vz/template/cache/centos-6.0-standard_6.0-1_amd64.tar.gz vcluster1b.int:/tmp
centos-6.0-standard_6.0-1_amd64.tar.gz 100% 203MB 101.5MB/s 00:02
real 0m1.896s
user 0m1.076s
sys 0m0.558s
even on an old Paxville DP (2.8 GHz) there's still something to gain:
vnode1:~# time scp -c blowfish /var/lib/vz/template/cache/centos-6.0-standard_6.0-1_amd64.tar.gz vnode2.int:/tmp
centos-6.0-standard_6.0-1_amd64.tar.gz 100% 203MB 29.0MB/s 00:07
real 0m6.703s
user 0m3.474s
sys 0m1.218s
vnode1:~# time scp -c arcfour /var/lib/vz/template/cache/centos-6.0-standard_6.0-1_amd64.tar.gz vnode2.int:/tmp
centos-6.0-standard_6.0-1_amd64.tar.gz 100% 203MB 67.7MB/s 00:03
real 0m3.363s
user 0m1.939s
sys 0m0.891s
I guess the rsync devolopers knows what they do, so why do the use a slow digest for error detection (i just wonder)?
As I mentioned, for unsecure networks this might be a good option to sacrifice performance for security. But why use it in a secure environment where you really need all the performance you can get. And by using the older protocol MD4 (which rsync used prior to v. 3) you do get the benefit of better performance and also less load on the actual process.
We cannot assume that the network is secure, so the best we can do is to make that configurable.
Yes, just as we have proposed earlier...
As e100 suggested earlier. Add it to the manager where you could easily have some options to change this your self.
Options that says.
1. If you are using a secure management/storage network then maybe rsync (with possibility to set your own options)
2. If you are using unsecure management/storage network, choose cipher and protocol as desired.
IMHO this is a very dangerous option (so this is not my top priority). Using ssh 'none' cipher would be more appropriate, but AFAIK that is still not implemented.
I believe the none cipher is possible with a custom SSH client/server.
In my case I found a way to utilize my CPU better and accelerate AES ciphers.
What we are missing is a simple way to define what cipher to use.
Maybe an option in /etc/pve/datacenter.cfg like:
ciphers: arcfour256,blowfish
And use arcfour256 by default (or is that too insecure?)
My personal feeling is that RC4 is good enough for error detection. But I do not really want an old protocol (instead just use RC4 digest). So what other side effects does the '--protocol 28' option has?
I like the way you mentioned to configure it.
Maybe blowfish should be the default, I believe arcfour is less secure and a less secure default does not seem like a good idea.
If people want to be less secure to get more speed they should make that decision on their own.
Just a simple way to configure this should be enough. Seems like there is no "golden setting" here. But you should be able to easily change these settings so that you can tune for your cluster/setup along with an short explanation to why. I am sure that many others would also prefer if their VM migrations could be done in less then 20-30% of time from what the default values offer.
Hello There,
I stumbled upon this thread while looking for a way to reduce the encryption bottleneck for scp on my storage network. This is precisely what I am looking for but this thread is five years old. Does anyone know the current state of OpenSSH and Rsync on PVE? Have any of these suggestions been implemented? Could we please have an update Dietmar?
I was just about to try setting "Cipher none" in ~/.ssh/config along with turning off compression for my Infiniband storage network as well as my 10G baseT, which is straight piped between my two mightiest hosts. Neither network is physically accessible to badguys as they are not trunked to the production LAN.
I have not yet attempted to change the Cipher since it needs a reboot apparently but with the default Cipher I am able to get about 450 MB/s before the single thread saturates one of the host's cores. My faster zpools can handle more than double this throughput and my IPoIB network can handle more than that.
We are trying to NOT have a SAN this time around and have a number of >1TB images that need to get moved around from host to host for maintenance, etc. It would be ideal to do away with encryption entirely on the isolated networks but I would settle for being able to change it to the fastest available cipher since whichever cipher it is using by default can saturate a core on our brand new CPUs.
Intel E5 2640 V4 @ 2.4 GHz
Sincerely,
GB
I stumbled upon this thread while looking for a way to reduce the encryption bottleneck for scp on my storage network. This is precisely what I am looking for but this thread is five years old. Does anyone know the current state of OpenSSH and Rsync on PVE? Have any of these suggestions been implemented? Could we please have an update Dietmar?
I was just about to try setting "Cipher none" in ~/.ssh/config along with turning off compression for my Infiniband storage network as well as my 10G baseT, which is straight piped between my two mightiest hosts. Neither network is physically accessible to badguys as they are not trunked to the production LAN.
I have not yet attempted to change the Cipher since it needs a reboot apparently but with the default Cipher I am able to get about 450 MB/s before the single thread saturates one of the host's cores. My faster zpools can handle more than double this throughput and my IPoIB network can handle more than that.
We are trying to NOT have a SAN this time around and have a number of >1TB images that need to get moved around from host to host for maintenance, etc. It would be ideal to do away with encryption entirely on the isolated networks but I would settle for being able to change it to the fastest available cipher since whichever cipher it is using by default can saturate a core on our brand new CPUs.
Intel E5 2640 V4 @ 2.4 GHz
Sincerely,
GB
Why does changing the Cipher needs a reboot? I think a SSH server restart is sufficient.
You then change all your ~/.ssh/config for the specific hosts.
You then change all your ~/.ssh/config for the specific hosts.
LnxBil, you are quite correct. I misspoke based on a forum post I read on the subject. I messed around with it a bit yesterday and found that restarting ssl was good enough to make whatever is in the .ssh/config active. What I found to be a bit mysterious was that certain of the encryption algos are not accepted such as arcfour128 although it is listed in the "Ciphers" line of the config. FWIW, the fastest one for me was
aes128-gcm@openssh.com
It could do 512 MB/s vs aes128-ctr, which churned at a rate of 450 MB/s.
Fabian, turning off encryption entirely is by far the best option for me! Thank you! I am going to give this a whirl today and will report back to the class.
GB
aes128-gcm@openssh.com
It could do 512 MB/s vs aes128-ctr, which churned at a rate of 450 MB/s.
Fabian, turning off encryption entirely is by far the best option for me! Thank you! I am going to give this a whirl today and will report back to the class.
GB