Open file limits on unprivileged LXC

[gmbeniamin]

We are hosting multiple LXC containers on our dedicated servers (Debian 10). We are mainly hosting game servers and we noticed an oddly annoying pattern.

Our game servers get the following errors and shut down. It happens randomly and when there are most players on each servers.

terminate called after throwing an instance of 'boost::wrapexcept<boost::system::system_error>'
  what():  boost::random_device: cannot open random-number pseudo-device /dev/urandom: No file descriptors available

and



How can we set the limit to a higher number for each lxc unprivileged container? We looked into other tutorials but we could not make it work. Does the container take in consideration the host for this as well? Do we need to increase the open file limit to the dedicated server OS and the lxc unprivileged container?

We have an experience of over 2 years using proxmox, but we mostly used the interface and have limited knowledge on the server side of things.

 

[oguz]

hi,

you can set the config option for the container: lxc.prlimit.nofile: 65536 and restart the container.

then when you run ulimit -a in the container you'll see the limit for open files increased

 

[gmbeniamin]

EDIT: I forgot to mention this solutions works great and we added the option to all our lxc containers configuration files.*

Is it possible to add this option to each newly created LXC container, without the need to manually add the option?

 

[yswery]

Is it possible to add this option to each newly created LXC container, without the need to manually add the option?

You could apply it on

/usr/share/lxc/config/common.conf

this way its applied to all LXC containers wide

 

[gmbeniamin]

I checked the file and noticed lxc.pty.max = 1024. Does it have something to do with with the open files? I am just asking because I noticed the word pseudo in my first error.

In /usr/share/lxc/config/common.conf I would just need to add lxc.prlimit.nofile: 65536? It seems way to simple and I do not want to cause any ruckus while in production.

 

[yswery]

In /usr/share/lxc/config/common.conf I would just need to add lxc.prlimit.nofile: 65536? It seems way to simple and I do not want to cause any ruckus while in production.

The way the LXC configs are loaded:

First you have /usr/share/lxc/config/common.conf and then what ever is in /usr/share/lxc/config/common.conf.d/* and after the OS specific for example /usr/share/lxc/config/debain.common.conf and after that your VMID.conf. thats the order of how its loaded if you look at the files.

you can create a new file in /usr/share/lxc/config/common.conf.d/prlimit.conf for example and add in there
lxc.prlimit.nofile: 65536. This WILL get overriden if you have in your VMID.conf file something different for that same config var though.

Hope that makes sense

 

[gmbeniamin]

I changed the VMID.conf file yesterday and added the lxc.prlimit.nofile: 65536 option. After I read your answer I created a file in /usr/share/lxc/config/common.conf.d/prlimit.conf <-- this is the file. When I tried stopping the running LXC container I got the following error:

lxc-stop: 133: confile.c: parse_line: 2684 Invalid argument - Invalid configuration line: lxc.prlimit.nofile: 65536
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/usr/share/lxc/config/common.conf.d//prlimit.conf" at line "lxc.prlimit.nofile: 65536"
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/usr/share/lxc/config/common.conf" at line "lxc.include = /usr/share/lxc/config/common.conf.d/"
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/usr/share/lxc/config/debian.common.conf" at line "lxc.include = /usr/share/lxc/config/common.conf"
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/var/lib/lxc/133/config" at line "lxc.include = /usr/share/lxc/config/debian.common.conf"
Failed to load config for 133
lxc-stop: 133: tools/lxc_stop.c: main: 145 Error opening container
command 'lxc-stop -n 133 --kill' failed: exit code 1

Is this due the fact that the VMID.conf file already has this option?

EDIT: I also noticed the limit does not apply for other users. When using ulimit -a on the root user it shows the correct amount of open files. When using the same command on a different user it shows 1024. How could I increate the user open files limit as well?

 

[gmbeniamin]

When accessing the container via SSH (Termius) all the ulimit -a shows only 1024. When using ulimit -Hn we see the correct number of files but ulimit -Sn shows only 1024.

Do we have to change the limits on the host node (dedicated server) as well?

 

[mathx]

I changed the VMID.conf file yesterday and added the lxc.prlimit.nofile: 65536 option. After I read your answer I created a file in /usr/share/lxc/config/common.conf.d/prlimit.conf <-- this is the file. When I tried stopping the running LXC container I got the following error:

lxc-stop: 133: confile.c: parse_line: 2684 Invalid argument - Invalid configuration line: lxc.prlimit.nofile: 65536
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/usr/share/lxc/config/common.conf.d//prlimit.conf" at line "lxc.prlimit.nofile: 65536"
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/usr/share/lxc/config/common.conf" at line "lxc.include = /usr/share/lxc/config/common.conf.d/"
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/usr/share/lxc/config/debian.common.conf" at line "lxc.include = /usr/share/lxc/config/common.conf"
lxc-stop: 133: parse.c: lxc_file_for_each_line_mmap: 132 Failed to parse config file "/var/lib/lxc/133/config" at line "lxc.include = /usr/share/lxc/config/debian.common.conf"
Failed to load config for 133
lxc-stop: 133: tools/lxc_stop.c: main: 145 Error opening container
command 'lxc-stop -n 133 --kill' failed: exit code 1

Is this due the fact that the VMID.conf file already has this option?

Yep breaks everything. This instruction is wrong and dangerous. (PVE 7.1 here)