too low open file limit

[asmar]

Hi all,

I'm installing a log software in one of my KVM VM with plenty of RAM and disk space but getting the following error:

There are ElasticSearch nodes in the cluster that have a too low open file limit. (below 64000) This will be causing problems that can be hard to diagnose. Read how to raise the maximum number of open files in the documentation.

In the VM it seems that the open file limit is not an issue:

root@logserver1:~/graylog2-setup# cat /proc/sys/fs/file-max
818835

Where do I need to set this limit? I searched in various places but no luck.

Any help is much appreciated.

 

[udo]

Hi all,

I'm installing a log software in one of my KVM VM with plenty of RAM and disk space but getting the following error:

There are ElasticSearch nodes in the cluster that have a too low open file limit. (below 64000) This will be causing problems that can be hard to diagnose. Read how to raise the maximum number of open files in the documentation.

In the VM it seems that the open file limit is not an issue:

root@logserver1:~/graylog2-setup# cat /proc/sys/fs/file-max
818835

Where do I need to set this limit? I searched in various places but no luck.

Any help is much appreciated.

Hi,
I'm not sure, but i guess the error messages mean "/proc/sys/fs/nr_open", which you can set via sysctl (fs.nr_open).
See "sysctl -a"

Udo

 

[asmar]

But the value of fs.nr_open is higher than what the program reports, see below:

root@logserver1:~# sysctl -a | grep open
fs.nr_open = 1048576

Is there anywhere a config at node level to set this value in each VM?

Thanks

 

[spirit]

/etc/security/limits.conf

 

[WaltervdSchee]

Yup, like spirit suggested, the user that you are using to run your log server application needs to be listed in the /etc/security/limits.conf with a large enough 'nofile' soft and hard limit.
Otherwise it uses the system default, which is 1024.

so, lets say the application runs under a user account 'logadmin'

/etc/security/limits.conf:

logadmin soft nofile 1024000
logadmin hard nofile 1024000

 

[asmar]

The same error occurs when I'm trying to dump some mysql dbs in another VM during a backup. I assume I need to add a similar entry for mysql user under limits.conf?

Is there a way to raise this number for any user running on VMs to avoid each time adding a user?

Thanks

 

[Ovidiu]

Very similar situation. Trying to run maldet which needs a way higher limit. It tries to raise it itself but fails. Manually edited /etc/security/limits.conf as shown below, do I need to run some command to apply the changes? Running it as root for testing.

maldet(29830): {mon} set inotify max_user_watches to 65536
/usr/local/maldetect/internals/functions: line 1448: /proc/sys/fs/inotify/max_user_watches: Read-only file system

Nov 15 11:44:00 alfred inotifywait[30212]: Setting up watches. Beware: since -r was given, this may take a while!
Nov 15 11:44:00 alfred inotifywait[30212]: Failed to watch /var/www/clients/client1/web2/web; upper limit on inotify watches reached!
Nov 15 11:44:00 alfred inotifywait[30212]: Please increase the amount of inotify watches allowed per user via `/proc/sys/fs/inotify/max_user_watches'.

root soft nofile 65535
root hard nofile 65535

sysctl -a | grep open
fs.nr_open = 1048576

cat /proc/sys/fs/inotify/max_user_watches
8192

cat /proc/sys/fs/file-max
3273989

I tried adding a new limit inside sysctl.conf but cannot commit the changes:
fs.inotify.max_user_watches=100000

sysctl -p
sysctl: setting key "fs.inotify.max_user_watches": Read-only file system

 

[sigxcpu]

@Ovidiu: do it in hypervisor. You cannot alter it inside LXC/OpenVZ.

 

[Ovidiu]

Ah! I understand! Does this mean that all settings of the hypervisor are inherited by the LXC? Could you have a LXC with different settings? My problem is solved, I'm just curios

 

[mir]

Ah! I understand! Does this mean that all settings of the hypervisor are inherited by the LXC? Could you have a LXC with different settings? My problem is solved, I'm just curios

No, all LXC shares the same runtime with the host.