[SOLVED] lxc_init: 845 Failed to run lxc.hook.pre-start for container

[space_people]

We've got several containers failing to start at host reboot (they are configured to start at boot, any order). Each of them gives an identical error in the UI (aside from container #):

run_buffer: 571 Script exited with status 2
lxc_init: 845 Failed to run lxc.hook.pre-start for container "XXX"
__lxc_start: 2046 Failed to initialize container "XXX"
TASK ERROR: startup for container 'XXX' failed

However, the affected containers will start manually afterward. These containers include nvidia-device passthroughs, root UID/GID remapping, and a mountpoint for /tank1/dataset1

A similarly-configured container include the same customizations, but the mountpoint for /tank2/dataset2, and that container starts just fine at boot.

We've tried downgrading pve-container (6.1.4 --> 6.0.18), as was mentioned in a few posts about similar startup issues, but that did not remedy, so it was upgraded back to current.

We've tried the remedies for starting ZFS before pvestatd, but that didn't fix it, so those changes were undone.

Any ideas for us to try?

 

[Impact]

Hard to say without debug logs or the config but for the GPU try adding this in your crontab

@reboot /usr/bin/nvidia-smi > /dev/null

For the mount issue try this: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#first_guest_boot_delay

 

[space_people]

Hard to say without debug logs or the config but for the GPU try adding this in your crontab

@reboot /usr/bin/nvidia-smi > /dev/null

For the mount issue try this: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#first_guest_boot_delay

Which logs would we be looking into?

What exactly does writing nvidia-smi to /dev/null achieve?

We should state, the boot delay had been tried previously (either 30 or 60 seconds), with no success.

However, commenting out the mountpoint for /tank1/dataset1 in the container config did eliminate the error.

 

[Impact]

journalctl -r and these: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_obtaining_debugging_logs
You can also set the debug option for the CT.
Running nvidia-smi makes sure that the NVIDIA devices are created.
Please don't use quotes without need.

 

[gfngfn256]

Maybe a workaround:

Find an LXC that does starts up fine at boot & then configure the order for that LXC to startup first, & then add a Startup delay to that container, so that the subsequent LXCs startup successfully. You could even create a "bogus" LXC just for this task.

 

[space_people]

Must have been mistaken, the 10 second boot delay fixed the issue for us. Thanks for your help today!

 

[Impact]

@gfngfn256 But why? The First Guest Boot Delay is made exactly for this.

 

[gfngfn256]

The First Guest Boot Delay is made exactly for this.

I thought (maybe incorrectly?) that the OP was having issues with the First Guest Boot Delay not helping his issue.

 

[space_people]

I thought (maybe incorrectly?) that the OP was having issues with the First Guest Boot Delay not helping his issue.

The GUI has a Startup delay that can be set in Options > Start/Shutdown order for the CT; it was tried previously (either 30 or 60 seconds ?), but the startup error was not fixed. Not sure why that didn't work and First Guest Boot Delay did. In any case, it seems to have resolved.

 

[leesteken]

The GUI has a Startup delay that can be set in Options > Start/Shutdown order for the CT; it was tried previously (either 30 or 60 seconds ?), but the startup error was not fixed. Not sure why that didn't work

That delay is between this CT/VM and the next. It's not a delay before your CT/VM. See also the manual: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pct_startup_and_shutdown