CT with exceptionally high cpu usage- where to look?

Saahib

Member
May 2, 2021
84
3
13
Have two identical servers.
1. One with proxmox 8.x (lets call it CT-A)
2. One with Ubuntu 20.04 LTS (And call this Dedi-B)

Have a web based app running nginx+php-fpm (mysql is on other server).
Now problem is that if I use LXC CT (CT-A) on for that App on proxmox server with same amount as host node and plenty of RAM (our web app needs really small ram) then CT is eating all CPU and constantly doing 100%, while if we send same traffic with same (nginx+ fpm) config to 2nd server (Dedi-B ie. Dedicated ubuntu server), there is hardly 2-5% cpu usage.

Both servers are on same DC, exactly same HW config, remote mysql server is also in same DC. Have tried really lots of combination but just con't find where is the issue. All servers are communicating with each other on private lan and have full 1 gbit link. Mysql server has no load issue to rule out if sql might causing slow responses,

There are no errors in PHP logs, fpm or nginx logs either.

I know that there could be some performance loss when using virtualization but should be neglible. Here I am not even sure if its virtualization or something else.

Suggestion are welcome to point to right direction or where to start looking to find exact reason for the load. I am ready to start with basic troubleshooting as that would eliminate the chances of human error.
 
Lots of php-fpm process which is expected. Tried php-fpm optimization, no gain. Now situation is this that webserver (CT-A) with virtualization can't handle more than 100 concurrent connections while other one (Dedi-B) is just fine with more than 500 concurrent connection
 
What OS in the CT? Is perhaps I/O the problem (iotop)? What filesystem/storage do you use? Does your app run in Docker? I mean, all the usual ways of tracking a performance problem apply here.
 
Last edited:
CT OS is ubuntu 20.04, there is no docker there, its simple nginx with php-fpm in ct. There is nothing iotop, almost no disk io. Here I am attaching normal htop, its mostly like this. This is when have limited traffic to this ct so that it won't get overloaded.

1695108983276.png
 
LXC containers have very little overhead compared to "native". So maybe there is something not directly related to Proxmox such as the various kernel mitigations for side-channel attacks. Those are known to reduce performance and maybe they are enabled on one machine but not the other. You might try booting Proxmox with "mitigations=off" on the kernel command line.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!