We've noticed a significant performance difference between 32-bit VM's and 64-bit VM's running as non-KVM hosts on Proxmox 1.5 2.6.24-10 PVE Kernel.
Origin of the problem - We noticed that in our company development efforts a software build was taking ~8 minutes to complete on a laptop but nearly ~45 minutes to complete on a comparable generation server hardware. So that lead to further testing on the server and we found the following:
:~# uname -a
Linux Proxmox-HOST 2.6.24-10-pve #1 SMP PREEMPT Tue Dec 29 10:16:15 CET 2009 x86_64 GNU/Linux
:~# pveperf
CPU BOGOMIPS: 30402.90
REGEX/SECOND: 418389
HD SIZE: 18.21 GB (/dev/mapper/pve-root)
BUFFERED READS: 17.68 MB/sec
AVERAGE SEEK TIME: 28.62 ms
FSYNCS/SECOND: 103.10
DNS EXT: 116.71 ms
DNS INT: 3.06 ms (contoso.com)
:~# openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 287 4096 bit private RSA's in 9.74s
Doing 4096 bit public rsa's for 10s: 20193 4096 bit public RSA's in 9.76s
OpenSSL 0.9.8g 19 Oct 2007
built on: Mon Jan 11 18:18:01 UTC 2010
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wa,--noexecstack -g -Wall -DMD32_REG_T=int -DMD5_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.033937s 0.000483s 29.5 2069.0
From a 64-bit Debian Lenny guest on Proxmox-HOST:
64-bit-guest:~# openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 303 4096 bit private RSA's in 9.94s
Doing 4096 bit public rsa's for 10s: 21977 4096 bit public RSA's in 9.93s
OpenSSL 0.9.8g 19 Oct 2007
built on: Mon Jun 7 19:28:26 UTC 2010
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wa,--noexecstack -g -Wall -DMD32_REG_T=int -DMD5_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.032805s 0.000452s 30.5 2213.2
From a 32-bit Debian Lenny guest on Proxmox-HOST:
32-bit-guest:~# openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 105 4096 bit private RSA's in 10.06s
Doing 4096 bit public rsa's for 10s: 7355 4096 bit public RSA's in 10.00s
OpenSSL 0.9.8g 19 Oct 2007
built on: Tue Jun 8 21:38:47 UTC 2010
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -march=i686 -Wa,--noexecstack -g -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DSHA1_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.095810s 0.001360s 10.4 735.5
Note: I've performed the same tests on Proxmox 1.6 on a newer version of the 2.6.24-12 PVE Kernel with the same results, and on both AMD and Intel hardware with similar yet proportionally different results - the Intel HW we have is newer and therefore faster than the AMD - but the result set is similar proportionally - ~1/3 as fast.
If there is other information that I can provide that would be helpful please let me know.
Cheers,
Jeff Hahn
Origin of the problem - We noticed that in our company development efforts a software build was taking ~8 minutes to complete on a laptop but nearly ~45 minutes to complete on a comparable generation server hardware. So that lead to further testing on the server and we found the following:
:~# uname -a
Linux Proxmox-HOST 2.6.24-10-pve #1 SMP PREEMPT Tue Dec 29 10:16:15 CET 2009 x86_64 GNU/Linux
:~# pveperf
CPU BOGOMIPS: 30402.90
REGEX/SECOND: 418389
HD SIZE: 18.21 GB (/dev/mapper/pve-root)
BUFFERED READS: 17.68 MB/sec
AVERAGE SEEK TIME: 28.62 ms
FSYNCS/SECOND: 103.10
DNS EXT: 116.71 ms
DNS INT: 3.06 ms (contoso.com)
:~# openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 287 4096 bit private RSA's in 9.74s
Doing 4096 bit public rsa's for 10s: 20193 4096 bit public RSA's in 9.76s
OpenSSL 0.9.8g 19 Oct 2007
built on: Mon Jan 11 18:18:01 UTC 2010
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wa,--noexecstack -g -Wall -DMD32_REG_T=int -DMD5_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.033937s 0.000483s 29.5 2069.0
From a 64-bit Debian Lenny guest on Proxmox-HOST:
64-bit-guest:~# openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 303 4096 bit private RSA's in 9.94s
Doing 4096 bit public rsa's for 10s: 21977 4096 bit public RSA's in 9.93s
OpenSSL 0.9.8g 19 Oct 2007
built on: Mon Jun 7 19:28:26 UTC 2010
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wa,--noexecstack -g -Wall -DMD32_REG_T=int -DMD5_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.032805s 0.000452s 30.5 2213.2
From a 32-bit Debian Lenny guest on Proxmox-HOST:
32-bit-guest:~# openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 105 4096 bit private RSA's in 10.06s
Doing 4096 bit public rsa's for 10s: 7355 4096 bit public RSA's in 10.00s
OpenSSL 0.9.8g 19 Oct 2007
built on: Tue Jun 8 21:38:47 UTC 2010
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -march=i686 -Wa,--noexecstack -g -Wall -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DSHA1_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
sign verify sign/s verify/s
rsa 4096 bits 0.095810s 0.001360s 10.4 735.5
Note: I've performed the same tests on Proxmox 1.6 on a newer version of the 2.6.24-12 PVE Kernel with the same results, and on both AMD and Intel hardware with similar yet proportionally different results - the Intel HW we have is newer and therefore faster than the AMD - but the result set is similar proportionally - ~1/3 as fast.
If there is other information that I can provide that would be helpful please let me know.
Cheers,
Jeff Hahn
