Super-slow web interface performance

[ron]

Hi all,
my proxmox's web interface is so slow that it usually takes more then a minute to receive the requested web-page.
Proxmox runs on a dual 1Ghz Xeon Intel server with 2 SCSI HDs connected to a true hardware RAID-1.
current load average: 0.76, 1.18, 1.03.
this is usually the case unless I run ZGREP, for example, then the load average can soar up to 17.something.something.
any thoughts?...

 

[tom]

Hi all,
my proxmox's web interface is so slow that it usually takes more then a minute to receive the requested web-page.
Proxmox runs on a dual 1Ghz Xeon Intel server with 2 SCSI HDs connected to a true hardware RAID-1.
current load average: 0.76, 1.18, 1.03.
this is usually the case unless I run ZGREP, for example, then the load average can soar up to 17.something.something.
any thoughts?...

hi,
too less physical memory? we recommend 512 mb at least.

 

[ron]

Forgot to mention it but the server has 512 MB of RAM.
Also forgot to mention that I installed a self signed SSL certificate for the web interface website, if thats got anything to do with the slowness...

 

[tom]

Forgot to mention it but the server has 512 MB of RAM.
Also forgot to mention that I installed a self signed SSL certificate for the web interface website, if thats got anything to do with the slowness...

hi ron,
please check the start page to see the Physical Memory status
how many emails do you process per day, did you enable OCR (if yes, disable it for testing)

If you want, we can check your installation via SSH login (please open your firewall for our IP 213.129.239.114)
Please send the login credential to t.huber@proxmox.com

 

[ron]

Physical Memory (503MB/251MB)
Swap Space (1023MB/300MB)
HD Space (33428MB/5970MB)

OCR is on, we process about 18,000 Emails (legit and spam) every day, a total of 558,000 every month. Emails are being sent (outbound) and received (inbound) very quickly, it's just the web interface which performs poorly so I don't know if it's server-load related.

 

[tom]

Physical Memory (503MB/251MB)
Swap Space (1023MB/300MB)
HD Space (33428MB/5970MB)

OCR is on, we process about 18,000 Emails (legit and spam) every day, a total of 558,000 every month. Emails are being sent (outbound) and received (inbound) very quickly, it's just the web interface which performs poorly so I don't know if it's server-load related.

did you disable OCR - is it going better?
btw, in my opinion disabling OCR will not result in much slower spam detection. we have also other filtering rules covering these picture spam messages.

 

[ron]

hi ron,
please check the start page to see the Physical Memory status
how many emails do you process per day, did you enable OCR (if yes, disable it for testing)

If you want, we can check your installation via SSH login (please open your firewall for our IP 213.129.239.114)
Please send the login credential to t.huber@proxmox.com

Hi Tom,
does your offer still stand? we're just about to give up on that server.

 

[tom]

Hi Tom,
does your offer still stand? we're just about to give up on that server.

hi ron,
sure. we check your system tomorrow, just open the ssh access for our ip and send the credentials (send to t.huber@proxmox.com)

 

[tom]

Hi Tom,
does your offer still stand? we're just about to give up on that server.

Hi ron,

I just did a short analysis, your server reiceives over 95 % spam messages. You did not use our sophisticated greylisting algorith and spf, so all messages are accepted which results in a heavy load and a slow web interface. I enabled greylisting and SPF and we will review in depth tomorrow. with greylisting combined with spf lot of spam messages are rejected before they reach your network.

final solution:
- greylisting, spf
- use 'verify receivers' option to reject unknown receivers at SMTP (available in version 1.7) will be released on wednesday next week.
- if this does not help, upgrade RAM to 1 GB, or alternative, you can move to new hardware: a pentium III machine is since several years outdated.

enabling these settings could reduce your serverload by more than 50 %. I will contact you again, after final analysis.

 

[ron]

Hi Tom,
the reason I have disabled SPF and greylisting is because it was giving me to many false-positives and some important mails didn't reach their destination.
while minimizing greylisting's false positives is relatively easy (using greylist exclusions), SPF was giving me quite a headache because only the administrator on the sender side can resolve this situation (or am I wrong?..).

after you enabled greylisting and SPF, MTA and the webinterface were performing better, but now Emails are not being received nor sent and the webinterface is very slow again :-(

the thing is, while the webinterface was always performing poorly, mail flow was quick and reliable and over the past few days it is not. I wonder what went wrong as I did not change anything. I could re-install the system but then I will lose configuration (white list, Glist exclusions, rules order, etc) AND queued mails.

 

[tom]

Hi Tom,
the reason I have disabled SPF and greylisting is because it was giving me to many false-positives and some important mails didn't reach their destination.
while minimizing greylisting's false positives is relatively easy (using greylist exclusions), SPF was giving me quite a headache because only the administrator on the sender side can resolve this situation (or am I wrong?..).

after you enabled greylisting and SPF, MTA and the webinterface were performing better, but now Emails are not being received nor sent and the webinterface is very slow again :-(

the thing is, while the webinterface was always performing poorly, mail flow was quick and reliable and over the past few days it is not. I wonder what went wrong as I did not change anything. I could re-install the system but then I will lose configuration (white list, Glist exclusions, rules order, etc) AND queued mails.

Hi Ron,
we are currently analysing on your machine, I will give you the results in a few hours,

 

[tom]

Hi Tom,
the reason I have disabled SPF and greylisting is because it was giving me to many false-positives and some important mails didn't reach their destination.
while minimizing greylisting's false positives is relatively easy (using greylist exclusions), SPF was giving me quite a headache because only the administrator on the sender side can resolve this situation (or am I wrong?..)..

greylisting cannot produce false positives, as all mail server send again if they go a temorary failure (450). only trojans and other spammer are sending just one time.

SPF: if there is no SPF record is for a domain, nothing will happen. If there is a SPF record for a domain and the sender is not sending from the domain specified in the SPF record, it is definitely not the true sender and also good to block. please make sure that your proxmox can query a fast enough DNS server for SPF records.

but we disabled both as it was before, you can decide what to enable.

conclusion:
your cpu is too slow for this amount of spam messages and therefore it take too long to analyse. as a consequence, proxmox opens more parallel processes which consumes more memory. at the end, the machine is under heavy load. you can increase memory to fight against this, better would be a faster machine (P4 3 GHZ or higher)

We will release version 1.7 on wednesday, with receiver verification. By enabling this, proxmox accepts just emails from existing internal users which leads to a massive reduction on emails to be analysed.

I would suggest to test this and see the results. I think this could be a solution also for your existing hardware.

 

[ron]

Hi Tom,
things look much better now, mail flow is fast and web-interface is performing better. I have re-enabled SPF & Greylisting and so far so good...

I would like to thank you and your associates for a world-class support, you have an excellent product!

Ron