Big performance problem

[giner]

Is there something unusual in dmesg, /var/log/messages or /var/log/syslog?
Are you able to boot from livecd on this hardware and check performance?

 

[bubbafish]

Nothing - really nothing unusual. Tried all offline tests - in theory the machine should be really fast - in fact it ain't.
Think there's no help anymore - I'll migrate the vms to my own server and reinstall the broken machine.

Thanks for your help.

 

[giner]

Just try to boot with, for example, the last Ubuntu LiveCD 10.10 and check performance.

 

[bubbafish]

As I wrote I've already performed those tests without result.

 

[dl_mj12]

Did the reinstallation help the 'broken' server?

We are experiencing very similar problems. pveperf gives great results, but rsync transfer speeds are <1-2mb/s

 

[bubbafish]

Hi,

reinstallation didn't fix the problem. Now I've temporarily changed hardware and everything runs fine but I've got to use the original server in a few days again.

 

[dl_mj12]

I think I've managed to isolate the problem to the Adaptec 5405 RAID Controller.... we're going to throw in a SATA HDD tonight and bypass the controller, install proxmox and test.
The problem is not present on another platform (proxmox1.6 also) which is running a different RAID controller, and IS present on a third system running the same Adaptec controller...

Not looking good for the Adaptec 5405 + rsync!!

I will come back and confirm the above, or any other findings.

 

[mdo]

Hi

I am curious whether you have any further results? You seem to suspect the issue caused by the Adaptec 5405 RAID controller and we are also suspicious about an rsync (only) issue in two systems that are using the Adaptec 5405 controllers.

My colleague described our problem here: http://forum.proxmox.com/threads/5999-rsync-very-slow
In our case it's only an rsync issue, everything else (cp, mv, etc.) is all good, normal speed.

On one of the two systems where we experience the rsync slow speed (Intel E5620, Adaptec 5405, 2 x 300GB SAS RAID1 plus 2 x 1TB Sata RAID1, 12 GB RAM), we connected a simple Sata disk straight onto the built-in Sata controller - and still experience the slow speed.

On the other hand, another lower-end (test) server (Intel X3430, 2 x 500GB Sata using the mainboard controller, same Proxmox version & kernel) runs rsync well at expected speed.

Michael

 

[bubbafish]

Yepp,

seems we've found the error. After changing memory (Kingston ECC) which ran without any errors on memtest rsync-speed seems to be fine (140-180 Mbytes/s). More results next week.

Kind regards.

EDIT: We've also tried 5405 without any problems.

 

[dswartz]

boy that is weird! only thing i can think of is ecc ram was generating a lot of errors and system was working hard to correct them and slowing you down?

 

[mdo]

Many Thanks for that feedback. it's hard to believe but we will definitely look at that asap here at our system and will let everyone know the results.

 

[dswartz]

Well, I am mystified as to problems with ECC memory that cause serious slowdown with no crashes or errors other than something weird like that...