pveproxy eats available ram

Thanks for the update!
the ss output looks like there where tons of pveproxy processes still running - ist this still the case?

also - the anonymization is understandable - but just that I get a clear picture - these were just regular IP-addresses? like e.g. 212.224.123.69

Thanks!
 
The ss output keeps growing of pveproxy processes every few minutes as long as /etc/default/pveproxy contains at least one IP restriction.
It reverts back to normal behavior when this file is removed, as a workaround, I don't think you need this output isn't it ?
Yes, these are just IPv4 public addresses.
 
The problem is I cannot reproduce the issue here (I created '/etc/default/pveproxy' like you have it, but with different IP addresses - it works as expected, without any hanging worker processes)...

Could I ask you to provide your (unchanged) '/etc/default/pveproxy' file - if you don't want to share it publicly - you can send it to me via e-mail (s.ivanov _at_ proxmox.com)

I prepared a patch which adds a few debug-statements to the method, which handles the allow_from and deny_form settings - would you be willing to add it in your environment and let pveproxy run with it for a short while?

The file is attached.

* Download it to the server
* Apply it by running:
Code: 
patch /usr/share/perl5/PVE/APIServer/AnyEvent.pm /path/to/0001-debug-add-debug-log-for-check_host_access.patch.txt
systemctl restart pveproxy

* It will print (quite a bit of) debug information to /var/log/pveproxy/access.log

after you've observed that a few of the hanging worker-processes have accumulated - stop pveproxy, collect the log (and provide it here or via e-mail to me), and reinstall libpve-http-server-perl to get rid of the debug-code:
Code: 
apt install --reinstall libpve-http-server-perl

Thanks for helping us finding the issue!
 

Attachments

Thanks! - just took a quick glance - and I think I see a problem in our code - will try to come up with a patch.
It would help to understand how this happens (then I could reproduce it and verify that the patch really solves the issue:
* Do you have some software in place, which connects regularly to 8006 on the box - e.g. for monitoring purposes?
* does anything else open connections and then close them directly?
 
Ohoh the answer is YES, Monitorix is installed, checking pveproxy process and port on a regular basis, I did not imagine it could harm this way...
 
You must log in or register to reply here.
Top Bottom