pvesr.service: Unit entered failed state.

[dafydd]

I have a server running 5.4-13 with the latest packages. It has 24 containers running on it. I've been migrating some older openvz containers to it, and they all are up and running and visible in the web interface. Shortly after starting up a newly transferred container. I was unable to run pct or pve commands.

# pct list
Unable to create new inotify object: Too many open files at /usr/share/perl5/PVE/INotify.pm line 397.

Stopping that container makes things work again. Here's the output of daemon.log after starting the CT

Oct 16 17:16:38 devhost pvedaemon[32554]: starting CT 163: UPID:devhost:00007F2A:095101B2:5DA74266:vzstart:163:root@pam:
Oct 16 17:16:38 devhost systemd[1]: Starting PVE LXC Container: 163...
Oct 16 17:16:39 devhost systemd-udevd[32578]: Could not generate persistent MAC address for vethMVNBXM: No such file or directory
Oct 16 17:16:39 devhost systemd[1]: Started PVE LXC Container: 163.
Oct 16 17:17:00 devhost systemd[1]: Starting Proxmox VE replication runner...
Oct 16 17:17:00 devhost pvesr[1571]: Unable to create new inotify object: Too many open files at /usr/share/perl5/PVE/INotify.pm line 397.
Oct 16 17:17:00 devhost systemd[1]: pvesr.service: Main process exited, code=exited, status=24/n/a
Oct 16 17:17:00 devhost systemd[1]: Failed to start Proxmox VE replication runner.
Oct 16 17:17:00 devhost systemd[1]: pvesr.service: Unit entered failed state.
Oct 16 17:17:00 devhost systemd[1]: pvesr.service: Failed with result 'exit-code'.
Oct 16 17:18:00 devhost systemd[1]: Starting Proxmox VE replication runner...

Not sure if that error about MAC address has anything to do with it?

 

[dafydd]

P.S. Stopping one other container stopped it but it failed again after re-starting the CT. I then experimenting by stopping another container and after re-starting, it still gave this error
Oct 16 18:45:00 devhost pvesr[19771]: Unable to create new inotify object: Too many open files at

But a couple of minutes later systemd restarted that service and it's back up and working.
It still seems strange for only 24 containers to cause the problem. I've had a server running 3.4 with more than 35 containers (most of them the same containers running on this new server)

 

[Stoiko Ivanov]

I think you might be running into https://bugzilla.proxmox.com/show_bug.cgi?id=1042

This has been resolved in PVE 6.0 (which ships a new enough kernel to support a higher default value for the inotify objects).
You could upgrade.

Alternatively you can also adapt the sysctl flags in your setup.

I hope this helps!

 

[dafydd]

Thanks for that! An upgrade to 6 is my next step but the plan is to move everything off the other server so I can install v6 from scratch on that box.