Ollama AI LLM in LXC

T.Herrmann said:
Is there any way to fix cores of one socket from dual socket system to a LXC?
Click to expand...
Only while starting the process or after it has been created. taskset is the tool to do that.

Upstream should fix the bug that is their program. Reading /proc/cpuinfo is not the way to detect how many processors can be used. They should use the syscall tha tools like *top use. They can read the number of processors correctly.
 
Top shows the same problem. If I adapt 8 cores it shows all cores of the two socket CPU's in the docker container but not in the LXC it self.
 
Hmm, you're right. That's strange. In the past, there were problems recognizing the correct number of processors in the LX(C) container (that was what I was refering to), yet I have never tried to check it in Docker running in LXC. I don't know how that is even possible.
 
Yes, but especially for LLM with large models that not fit into the GPU RAM a LXC with big EPYC 512 Core platform would be a very efficient way to use this as a side application instead of bind this resources by a VM which can't be used for other LXC/VM. LXC are really nice for over provisioning.
 
You must log in or register to reply here.
Top Bottom