LXCs Containers does not start their internal services after "abrupt" shutdown

[brunoverdi]

Hey guys, what's up?

There's a strange problem that happens with our Proxmox servers from the beginning when we started using version 6.0.
When there's a blackout or something that "abrupt" shutdown our Proxmox physical hosts; after we turned them on, all LXCs containers does not start their internal services. (For example: The lxc container does not start ssh, named or any other services that should be running inside it)

All of them is set to boot with the physical host.

To make it work, we need to safetly shutdown all LXCs containers, and start it again... Then all internal services start correctly.

We are using:



And we didn't see anything wrong in the logs.

Does anyone have any idea what might be happening?

Best Regards

 

[Moayad]

Hi,

after we turned them on, all LXCs containers does not start their internal services.

Have you tried to set a startup delay on your LXC containers?

Please post the LXC config pct config <CTID>

I recommend you upgrade the Proxmox VE to the latest version of PVE 6.4.x since we did some fixes on multiple packages.

 

[brunoverdi]

Hi,


Have you tried to set a startup delay on your LXC containers?

Please post the LXC config pct config <CTID>

I recommend you upgrade the Proxmox VE to the latest version of PVE 6.4.x since we did some fixes on multiple packages.

Hi..

1) How can we "delay" the LXCs containers startup?

2)



3) We are scheduling this, and in fact, we already have two Proxmox physical servers in this version 6.4.X, but we didn't test these LXCs containers on them.

Just one more question.... To upgrade from 6.2 to the last stable 6.4.4 (I guess) we need to do the apt update && apt full-upgrade? Cause we are afraidn to upgrade to the last one 7.0.... So if we do apt update && apt full-upgrade we are going to upgrade to the last stable version 6.4.4?

Thanks for the reply.

 

[Moayad]

Hi,

1) How can we "delay" the LXCs containers startup?

Under the:

Datacenter -> Node -> CTID -> Options -> Start/Shutdown order

Please try to set the startup delay [0] for the containers and see if they start the services after the restart.

3) We are scheduling this, and in fact, we already have two Proxmox physical servers in this version 6.4.X, but we didn't test these LXCs containers on them.

Just one more question.... To upgrade from 6.2 to the last stable 6.4.4 (I guess) we need to do the apt update && apt full-upgrade? Cause we are afraidn to upgrade to the last one 7.0.... So if we do apt update && apt full-upgrade we are going to upgrade to the last stable version 6.4.4?

Yes, you can upgrade to the latest version of Proxmox 6.4 by doing apt update && apt dist-upgrade [1]

[0] https://pve.proxmox.com/pve-docs/chapter-pct.html#pct_startup_and_shutdown
[1] https://forum.proxmox.com/threads/proxmox-ve-6-4-available.88336/

 

[brunoverdi]

Thanks for your help!
As our server is in production environment, we will schedule this upgrade and test the "delay" option too.

One of them is already in 6.4.4 version, but unfortunately this error with LXCs containers still persists.

Another question... Is there any way to see the logs from: lxc-start LXCID-F --logfile /tmp/LXCID.log --logpriority 90 command in web interface?
I mean.. When we start a LXC container from there, is there any configuration we can set to see all debug log there?
Can we change a command, or file to show this debug log in this web interface?


Best Regards

 

[Moayad]

Hi,

One of them is already in 6.4.4 version, but unfortunately this error with LXCs containers still persists.

Could you please share the output of journalctl -b from the CT container?

Another question... Is there any way to see the logs from: lxc-start LXCID-F --logfile /tmp/LXCID.log --logpriority 90 command in web interface?
I mean.. When we start a LXC container from there, is there any configuration we can set to see all debug log there?
Can we change a command, or file to show this debug log in this web interface?

on the GUI no that is not possible

 

[brunoverdi]

Hi,

Could you please share the output of journalctl -b from the CT container?

Do you want us to share this output after the problem happens? And most of our CTs are still CentOS 6..

We're gonna shedule a "forced shutdown" test in one of ours Proxmox Servers and try the options you described above.

Best Regards

 

[Moayad]

Do you want us to share this output after the problem happens? And most of our CTs are still CentOS 6..

Yes, if the issue occurs again please send the output of journalctl -b from one CT.