Unable to login via Web GUI; SSH works

[fabian]

FWIW, in almost all cases the only downside of using "apt upgrade" is that it might not upgrade all of our packages. we try hard to encode every interdependency on the package level, so usually this only means you run a less-well-tested and/or outdated combination of versions. of course it can happen that we miss adding such a dependency edge, in which case stuff might break as a result of a version mismatch somewhere. the symptoms here in this thread seem to have another cause though - although I am not sure which one exactly. replicating the behaviour on a test system while writing down the steps to arrive at it would probably help in finding out.

 

[esi_y]

FWIW, in almost all cases the only downside of using "apt upgrade" is that it might not upgrade all of our packages. we try hard to encode every interdependency on the package level, so usually this only means you run a less-well-tested and/or outdated combination of versions. of course it can happen that we miss adding such a dependency edge, in which case stuff might break as a result of a version mismatch somewhere. the symptoms here in this thread seem to have another cause though - although I am not sure which one exactly. replicating the behaviour on a test system while writing down the steps to arrive at it would probably help in finding out.

Thanks, Fabian. Hopefully the OP updates what happened after a reboot (which has not happened yet).

Do you agree that apt upgrade, when eventually dist-upgrade was run should not break the system at all or am I missing something myself?

(Any hint where the Unable to load access control list: No buffer space available is coming from? I checked possible parts including PVE::Tools and PVE::ProcFSTools in eval all the way to ipcc calls, but no luck guessing it. The postinst fails just because it cannot start pvestatd?)

 

[fabian]

"apt upgrade" if you are unlucky might cause some or all of our services and CLI tools to stop working (e.g., if it upgraded some library, but not the API code, or vice versa, and those need to be both upgraded in lockstep). this can only happen if we forget to add this version requirement in package dependencies though, which should hopefully be rather rare these days. such a partial upgrade problem is usually very easy to spot - it will cause all sorts of perl syntax errors in the most common case. of course there are other issues such as not actually getting all the bug or security fixes in such a case, and possible subtle behavioural differences that don't cause an outright failure. but this thread here smells to me like something messed up the system, and it only became obvious with the next upgrade because that attempted to restart the service. the upgrade was IMHO not the cause, but likely just what made the problem/misconfiguration/.. noticeable.

 

[cjbruck]

For anyone coming across this in the future, I never figured out the issue. I'm just having to reinstall on all 9 servers, being careful to use pveupdate and pveupgrade instead of apt update and apt upgrade.

 

[esi_y]

For anyone coming across this in the future, I never figured out the issue. I'm just having to reinstall on all 9 servers, being careful to use pveupdate and pveupgrade instead of apt update and apt upgrade.

Well you never even took a backup of a bad node to troubleshoot with debug flag on, unfortunately. It would have been possible to analyse some more.

 

[iwik]

Hi, I have found this post using "Unable to load access control list: No buffer space available". I was trying to enable Active directory auth in PVE. Seems we are missing some easy guide. I was not sure how and what to filter, so I keep it empty as in forum post. Preview showed some invalid usernames and groups. I have run sync, then pve showed error "No buffer space available" in during task. Now I am unable to login as root to web interface and even

pveum user list
Unable to load access control list: No buffer space available

Note: Big corporate AD with many entries. Can this be root cause?

EDIT: Seems /etc/pve/users.cfg reached 1 MB and is truncated. I have deleted it and now I am able to login.

Now how to get AD auth working..
https://forum.proxmox.com/threads/proxmox-ldap-sync-hard-limit-on-number-of-users.91320/#post-398786