Ollama AI LLM in LXC

[T.Herrmann]

Is there any way to fix cores of one socket from dual socket system to a LXC?

If have this problem with Ollama https://github.com/ollama/ollama/issues/5554

Thank you to all supporters.

 

[LnxBil]

Is there any way to fix cores of one socket from dual socket system to a LXC?

Only while starting the process or after it has been created. taskset is the tool to do that.

Upstream should fix the bug that is their program. Reading /proc/cpuinfo is not the way to detect how many processors can be used. They should use the syscall tha tools like *top use. They can read the number of processors correctly.

 

[T.Herrmann]

Top shows the same problem. If I adapt 8 cores it shows all cores of the two socket CPU's in the docker container but not in the LXC it self.

 

[LnxBil]

Hmm, you're right. That's strange. In the past, there were problems recognizing the correct number of processors in the LX(C) container (that was what I was refering to), yet I have never tried to check it in Docker running in LXC. I don't know how that is even possible.

 

[T.Herrmann]

Yes, but especially for LLM with large models that not fit into the GPU RAM a LXC with big EPYC 512 Core platform would be a very efficient way to use this as a side application instead of bind this resources by a VM which can't be used for other LXC/VM. LXC are really nice for over provisioning.