container is not running.

notter · Jun 4, 2021

Hi,

I didn't start container after reboot the server.

what do you suggest i do?

thanks for your interest

Code:

root@proxmox:~# pct start 106
Job for pve-container@106.service failed because the control process exited with error code.
See "systemctl status pve-container@106.service" and "journalctl -xe" for details.
command 'systemctl start pve-container@106' failed: exit code 1
root@proxmox:~#

----------

Code:

root@proxmox:~# systemctl status pve-container@106.service
● pve-container@106.service - PVE LXC Container: 106
   Loaded: loaded (/lib/systemd/system/pve-container@.service; static; vendor pr
   Active: failed (Result: exit-code) since Fri 2021-06-04 10:12:49 +03; 40s ago
     Docs: man:lxc-start
           man:lxc
           manct
  Process: 24419 ExecStart=/usr/bin/lxc-start -n 106 (code=exited, status=1/FAIL

Jun 04 10:12:49 proxmox lxc-start[24419]: lxc-start: 106: lxccontainer.c: wait_o
Jun 04 10:12:49 proxmox lxc-start[24419]: lxc-start: 106: tools/lxc_start.c: mai
Jun 04 10:12:49 proxmox lxc-start[24419]: lxc-start: 106: tools/lxc_start.c: mai
Jun 04 10:12:49 proxmox lxc-start[24419]: lxc-start: 106: tools/lxc_start.c: mai
Jun 04 10:12:49 proxmox systemd[1]: pve-container@106.service: Control process e
Jun 04 10:12:49 proxmox systemd[1]: pve-container@106.service: Killing process 2
Jun 04 10:12:49 proxmox systemd[1]: pve-container@106.service: Killing process 2
Jun 04 10:12:49 proxmox systemd[1]: Failed to start PVE LXC Container: 106.
Jun 04 10:12:49 proxmox systemd[1]: pve-container@106.service: Unit entered fail
Jun 04 10:12:49 proxmox systemd[1]: pve-container@106.service: Failed with resul
lines 1-18/18 (END)
[/ICODE]

----------
[CODE]root@proxmox:~# journalctl -xe
Jun 04 10:14:07 proxmox kernel: device veth106i0 entered promiscuous mode
Jun 04 10:14:07 proxmox kernel: eth0: renamed from vethT2GFUC
Jun 04 10:14:07 proxmox lxc-start[24659]: lxc-start: 106: lxccontainer.c: wait_o
Jun 04 10:14:07 proxmox lxc-start[24659]: lxc-start: 106: tools/lxc_start.c: mai
Jun 04 10:14:07 proxmox lxc-start[24659]: lxc-start: 106: tools/lxc_start.c: mai
Jun 04 10:14:07 proxmox lxc-start[24659]: lxc-start: 106: tools/lxc_start.c: mai
Jun 04 10:14:07 proxmox systemd[1]: pve-container@106.service: Control process e
Jun 04 10:14:07 proxmox systemd[1]: pve-container@106.service: Killing process 2
Jun 04 10:14:07 proxmox systemd[1]: pve-container@106.service: Killing process 2
Jun 04 10:14:07 proxmox systemd[1]: Failed to start PVE LXC Container: 106.
-- Subject: Unit pve-container@106.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit pve-container@106.service has failed.
--
-- The result is failed.
Jun 04 10:14:07 proxmox systemd[1]: pve-container@106.service: Unit entered fail
Jun 04 10:14:07 proxmox systemd[1]: pve-container@106.service: Failed with resul
Jun 04 10:14:07 proxmox pct[24657]: command 'systemctl start pve-container@106'
Jun 04 10:14:07 proxmox pct[24656]: <root@pam> end task UPIDroxmox:00006051:00
Jun 04 10:14:07 proxmox kernel: vmbr0: port 2(veth106i0) entered disabled state
Jun 04 10:14:07 proxmox kernel: device veth106i0 left promiscuous mode
Jun 04 10:14:07 proxmox kernel: vmbr0: port 2(veth106i0) entered disabled state
lines 1712-1735/1735 (END)

---------------------

Code:

root@proxmox:~# pveversion -v
proxmox-ve: 5.4-2 (running kernel: 4.15.18-30-pve)
pve-manager: 5.4-15 (running version: 5.4-15/d0ec33c6)
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.13.4-1-pve: 4.13.4-26
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-56
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-7
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-38
pve-container: 2.0-42
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-7
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-56
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

Stoiko Ivanov · Jun 4, 2021

The log messages are cut-off (and don't contain the interesting part).

Try starting the container with debug-logging - that should point to the problem:
https://pve.proxmox.com/pve-docs/chapter-pct.html#_obtaining_debugging_logs

notter · Jun 4, 2021

Please check it.

root@proxmox:~# lxc-start -n 106 -F -l DEBUG -o /tmp/lxc-106.log

Code:

lxc-start: 106: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106"
     lxc-start: 106: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106"
 lxc-start: 106: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106"
 lxc-start: 106: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106-1"
 lxc-start: 106: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-1"
 lxc-start: 106: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-1"
  lxc-start: 106: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106-2"
 lxc-start: 106: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-2"
 lxc-start: 106: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-2"
 lxc-start: 106: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106-3"
lxc-start: 106: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-3"
 lxc-start: 106: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-3"
 lxc-start: 106: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106-4"
  lxc-start: 106: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-4"
lxc-start: 106: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-4"
 lxc-start: 106: cgroups/cgfsng.c: mkdir_eexist_on_last: 1301 File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106-5"
lxc-start: 106: cgroups/cgfsng.c: container_create_path_for_hierarchy: 1353 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-5"
  lxc-start: 106: cgroups/cgfsng.c: cgfsng_payload_create: 1526 Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106-5"
lxc-start: 106: sync.c: __sync_wait: 62 An error occurred in another process (expected sequence number 7)
 lxc-start: 106: start.c: __lxc_start: 1989 Failed to spawn container "106"
                                      lxc-start: 106: tools/lxc_start.c: main: 330 The container failed to start
lxc-start: 106: tools/lxc_start.c: main: 336 Additional information can be obtained by setting the --logfile and --logpriority options
root@proxmox:~#

notter · Jun 4, 2021

root@proxmox:~# cat /tmp/lxc-106.log

Code:

lxc-start 106 20210604115455.506 INFO     lsm - lsm/lsm.c:lsm_init:50 - LSM security driver AppArmor
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:759 - Processing "reject_force_umount  # comment this to allow umount -f;  not recommended"
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:do_resolve_add_rule:505 - Set seccomp rule to reject force umounts
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:937 - Added native rule for arch 0 for reject_force_umount action 0(kill)

lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:966 - Added native rule for arch -1073741762 for reject_force_umount action 0(kill)
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:759 - Processing "[all]"
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:759 - Processing "kexec_load errno 1"
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:937 - Added native rule for arch 0 for kexec_load action 
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:946 - Added compat rule for arch 1073741827 for onfig_v2:966 - Added native rule for arch -1073741762 for finit_module action 327681(errno)
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:759 - Processing "delete_module errno 1"
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:937 - Added native rule for arch 0 for delete_module action 327681(errno)
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:946 - Added compat rule for arch 1073741827 for delete_module action 327681(errno)
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:956 - Added compat rule for arch 1073741886 for delete_module action 327681(errno)
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:966 - Added native rule for arch -1073741762 for delete_module action 327681(errno)
lxc-start 106 20210604115455.506 INFO     seccomp - seccomp.c:parse_config_v2:970 - Merging compat seccomp contexts into main context
lxc-start 106 20210604115455.506 INFO     conf - conf.c:run_script_argv:356 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "106", config section "lxc"
lxc-start 106 20210604115456.475 DEBUG    terminal - terminal.c:lxc_terminal_peer_default:714 - Using terminal "/dev/tty" as proxy
lxc-start 106 20210604115456.475 DEBUG    terminal - terminal.c:lxc_terminal_signal_init:192 - Created signal fd 9
lxc-start 106 20210604115456.475 DEBUG    terminal - terminal.c:lxc_terminal_winsz:90 - Set window size to 178 columns and 37 rows
lxc-start 106 20210604115456.792 INFO     start - start.c:lxc_init:906 - Container "106" is initialized
lxc-start 106 20210604115456.793 INFO     conf - conf.c:run_script:495 - Executing script "/usr/share/lxc/lxcnetaddbr" for container "106", config section "net"
lxc-start 106 20210604115457.754 DEBUG    network - network.c:instantiate_veth:206 - Instantiated veth "veth106i0/vethMFSY59", index is "25"
lxc-start 106 20210604115457.754 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1301 - File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106"
lxc-start 106 20210604115457.754 ERROR    cgfsng - cgroups/cgfsng.c:container_create_path_for_hierarchy:1353 - Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106"
lxc-start 106 20210604115457.754 ERROR    cgfsng - cgroups/cgfsng.c:cgfsng_payload_create:1526 - Failed to create cgroup "/sys/fs/cgroup/systemd//lxc/106"
lxc-start 106 20210604115457.754 ERROR    cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1301 - File exists - Failed to create directory "/sys/fs/cgroup/systemd//lxc/106-1"
lxc-start 106 20210604115457.754 ERROR    cgfsng - cgroups/cgfsng.c:container_create_path_for_hierarchy:1353 - Failed to create cgroup art 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "b *:* m"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 1:3 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 1:5 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 1:7 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 5:0 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 5:1 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 5:2 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 1:8 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 1:9 rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 136:* rwm"
lxc-start 106 20210604115457.781 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2476 - Set controller "devices.allow" set to "c 10:229 rwm"
lxc-start 106 20210604115457.781 INFO     cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2481 - Limits for the legacy cgroup hierarchies have been setup
lxc-start 106 20210604115457.782 INFO     start - start.c:do_start:1258 - Unshared CLONE_NEWCGROUP
lxc-start 106 20210604115457.783 DEBUG    storage - storage/storage.c:storage_query:253 - Detected rootfs type "dir"
lxc-start 106 20210604115457.783 DEBUG    conf - conf.c:lxc_mount_rootfs:1358 - Mounted rootfs "/var/lib/lxc/106/rootfs" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs" with options "(null)"
lxc-start 106 20210604115457.783 INFO     conf - conf.c:setup_utsname:817 - Set hostname to "server.lodybaby.com.tr"
lxc-start 106 20210604115457.809 DEBUG    network - network.c:setup_hw_addr:2772 - Mac address "56:53:C8:09:3F:E5" on "eth0" has been setup
lxc-start 106 20210604115457.809 DEBUG    network - network.c:lxc_setup_netdev_in_child_namespaces:3037 - Network device "eth0" has been setup
lxc-start 106 20210604115457.809 INFO     network - network.c:lxc_setup_network_in_child_namespaces:3058 - network has been setup
lxc-start 106 20210604115457.809 INFO     conf - conf.c:mount_autodev:1144 - Preparing "/dev"
lxc-start 106 20210604115457.809 INFO     conf - conf.c:mount_autodev:1191 - Prepared "/dev"
lxc-start 106 20210604115457.810 INFO     conf - conf.c:run_script_argv:356 - Executing script "/usr/share/lxcfs/lxc.mount.hook" for container "106", config section "lxc"
lxc-start 106 20210604115457.842 INFO     conf - conf.c:run_script_argv:356 - Executing script "/usr/share/lxc/hooks/lxc-pve-autodev-hook" for container "106", config section "lxc"
lxc-start 106 20210604115458.401 INFO     conf - conf.c:lxc_fill_autodev:1235 - Populating "/dev"
lxc-start 106 20210604115458.401 DEBUG    conf - conf.c:lxc_fill_autodev:1250 - Created device node "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/full"
lxc-start 106 20210604115458.402 DEBUG    conf - conf.c:lxc_fill_autodev:1250 - Created device node "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/null"
lxc-start 106 20210604115458.402 DEBUG    conf - conf.c:lxc_fill_autodev:1250 - Created device node "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/random"
lxc-start 106 20210604115458.402 DEBUG    conf - conf.c:lxc_fill_autodev:1250 - Created device node "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/tty"
lxc-start 106 20210604115458.402 DEBUG    conf - conf.c:lxc_fill_autodev:1250 - Created device node "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/urandom"
lxc-start 106 20210604115458.402 DEBUG    conf - conf.c:lxc_fill_autodev:1250 - Created device node "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/zero"
lxc-start 106 20210604115458.403 INFO     conf - conf.c:lxc_fill_autodev:1312 - Populated "/dev"
lxc-start 106 20210604115458.404 DEBUG    conf - conf.c:mount_entry:2053 - Remounting "/sys/fs/fuse/connections" on "/usr/lib/x86_64-linux-gnu/lxc/rootfs/sys/fs/fuse/connections" to respect bind or remount options
lxc-start 106 20210604115458.404 DEBUG    conf - conf.c:mount_entry:2074 - Flags for "/sys/fs/fuse/connections" were 4096, required extra flags are 0
lxc-start 106 20210604115458.404 DEBUG    conf - conf.c:mount_entry:2084 - Mountflags already were 4096, skipping remount
lxc-start 106 20210604115458.404 DEBUG    conf - conf.c:mount_entry:2128 - Mounted "/sys/fs/fuse/connections" on "/usr/lib/x86_64-linux-gnu/lxc/rootfs/sys/fs/fuse/connections" with filesystem type "none"
lxc-start 106 20210604115458.404 INFO     conf - conf.c:mount_file_entries:2359 - Finished setting up mounts
lxc-start 106 20210604115458.405 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1822 - Created directory for console and tty devices at "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/lxc"
lxc-start 106 20210604115458.405 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1867 - Mounted "/dev/pts/1" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/lxc/console"
lxc-start 106 20210604115458.406 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1875 - Mounted "/dev/pts/1" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/lxc/console"
lxc-start 106 20210604115458.406 DEBUG    conf - conf.c:lxc_setup_ttydir_console:1877 - Console has been setup under "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/lxc/console" and mounted to "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/console"
lxc-start 106 20210604115458.407 INFO     utils - utils.c:lxc_mount_proc_if_needed:1239 - I am 1, /proc/self points to "1"
lxc-start 106 20210604115458.572 WARN     conf - conf.c:lxc_setup_devpts:1642 - Invalid argument - Failed to unmount old devpts instance
lxc-start 106 20210604115458.573 DEBUG    conf - conf.c:lxc_setup_devpts:1679 - Mount new devpts instance with options "gid=5,newinstance,ptmxmode=0666,mode=0620,max=1024"
lxc-start 106 20210604115458.573 DEBUG    conf - conf.c:lxc_setup_devpts:1698 - Created dummy "/dev/ptmx" file as bind mount target
lxc-start 106 20210604115458.574 DEBUG    conf - conf.c:lxc_setup_devpts:1703 - Bind mounted "/dev/pts/ptmx" to "/dev/ptmx"
lxc-start 106 20210604115458.575 DEBUG    conf - conf.c:lxc_allocate_ttys:1015 - Created tty "/dev/pts/0" with master fd 11 and slave fd 14
lxc-start 106 20210604115458.576 DEBUG    conf - conf.c:lxc_allocate_ttys:1015 - Created tty "/dev/pts/1" with master fd 15 and slave fd 16
lxc-start 106 20210604115458.577 INFO     conf - conf.c:lxc_allocate_ttys:1031 - Finished creating 2 tty devices
lxc-start 106 20210604115458.577 DEBUG    conf - conf.c:lxc_setup_ttys:939 - Bind mounted "/dev/pts/0" onto "/dev/lxc/tty1"
lxc-start 106 20210604115458.578 DEBUG    conf - conf.c:lxc_setup_ttys:939 - Bind mounted "/dev/pts/1" onto "/dev/lxc/tty2"
lxc-start 106 20210604115458.578 INFO     conf - conf.c:lxc_setup_ttys:975 - Finished setting up 2 /dev/tty<N> device(s)
lxc-start 106 20210604115458.578 INFO     conf - conf.c:setup_personality:1742 - Set personality to "0x0"
lxc-start 106 20210604115458.578 DEBUG    conf - conf.c:setup_caps:2551 - Dropped mac_admin (33) capability
lxc-start 106 20210604115458.578 DEBUG    conf - conf.c:setup_caps:2551 - Dropped mac_override (32) capability
lxc-start 106 20210604115458.579 DEBUG    conf - conf.c:setup_caps:2551 - Dropped sys_time (25) capability
lxc-start 106 20210604115458.579 DEBUG    conf - conf.c:setup_caps:2551 - Dropped sys_module (16) capability
lxc-start 106 20210604115458.579 DEBUG    conf - conf.c:setup_caps:2551 - Dropped sys_rawio (17) capability
lxc-start 106 20210604115458.579 DEBUG    conf - conf.c:setup_caps:2554 - Capabilities have been setup
lxc-start 106 20210604115458.579 NOTICE   conf - conf.c:lxc_setup:3746 - The container "106" is set up
lxc-start 106 20210604115458.579 INFO     lsm - lsm/lsm.c:lsm_process_label_set_at:178 - Set AppArmor label to "lxc-106_</var/lib/lxc>//&:lxc-106_<-var-lib-lxc>:"
lxc-start 106 20210604115458.580 INFO     apparmor - lsm/apparmor.c:apparmor_process_label_set:1192 - Changed AppArmor profile to lxc-106_</var/lib/lxc>//&:lxc-106_<-var-lib-lxc>:
lxc-start 106 20210604115458.625 DEBUG    start - start.c:lxc_spawn:1853 - Preserved cgroup namespace via fd 20
lxc-start 106 20210604115458.627 NOTICE   utils - utils.c:lxc_setgroups:1408 - Dropped additional groups
lxc-start 106 20210604115458.627 NOTICE   start - start.c:start:2075 - Exec'ing "/sbin/init"
lxc-start 106 20210604115458.627 ERROR    start - start.c:start:2078 - No such file or directory - Failed to exec "/sbin/init"
lxc-start 106 20210604115458.629 ERROR    sync - sync.c:__sync_wait:62 - An error occurred in another process (expected sequence number 7)
lxc-start 106 20210604115458.630 WARN     network - network.c:lxc_delete_network_priv:2594 - Operation not permitted - Failed to remove interface "eth0" with index 25
lxc-start 106 20210604115458.630 DEBUG    network - network.c:lxc_delete_network:3185 - Deleted network devices
lxc-start 106 20210604115458.633 ERROR    start - start.c:__lxc_start:1989 - Failed to spawn container "106"
lxc-start 106 20210604115458.687 INFO     conf - conf.c:run_script_argv:356 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "106", config section "lxc"
lxc-start 106 20210604115459.190 INFO     conf - conf.c:run_script_argv:356 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "106", config section "lxc"
lxc-start 106 20210604115500.141 ERROR    lxc_start - tools/lxc_start.c:main:330 - The container failed to start
lxc-start 106 20210604115500.141 ERROR    lxc_start - tools/lxc_start.c:main:336 - Additional information can be obtained by setting the --logfile and --logpriority options
root@proxmox:~#

Stoiko Ivanov · Jun 4, 2021

notter said:
proxmox-ve: 5.4-2 (running kernel: 4.15.18-30-pve)

I just realized that you're still running PVE 5.4:
* PVE 5.x has been EOL for almost a year now - please consider upgrading as soon as possible, since you're missing quite a few bug-fixes and fixes for security-issues!
* this concrete issue has been addressed in newer versions and should not occur after an upgrade

I hope this helps!

notter · Jun 4, 2021

we have different error now.

root@proxmox:~# pct start 106

Code:

lxc_can_use_pidfd: 1905 Kernel does not support pidfds
__safe_mount_beneath_at: 1106 Function not implemented - Failed to open 48(dev)
__sync_wait: 36 An error occurred in another process (expected sequence number 7)
__lxc_start: 1999 Failed to spawn container "106"
startup for container '106' failed

root@proxmox:~# pveversion -v

Code:

proxmox-ve: 6.4-1 (running kernel: 4.15.18-30-pve)
pve-manager: 6.4-8 (running version: 6.4-8/185e14db)
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.13.4-1-pve: 4.13.4-26
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1

root@proxmox:~# pct list

Code:

can't open '/sys/fs/cgroup/blkio///lxc/101/blkio.throttle.io_service_bytes_recursive' - No such file or directory
root@proxmox:~#

Stoiko Ivanov · Jun 4, 2021

notter said:
proxmox-ve: 6.4-1 (running kernel: 4.15.18-30-pve)

you need to reboot in order to run the upgraded kernel.
In general you should reboot after each kernel-upgrade (but especially after a major distribution upgrade)

notter · Jun 6, 2021

Stoiko Ivanov said:
you need to reboot in order to run the upgraded kernel.
In general you should reboot after each kernel-upgrade (but especially after a major distribution upgrade)

I rebooted server after upgrade.
kernel upgraded however i have different error message
could you help us ?

thanks for your interest

Hi,

Code:

root@proxmox:~# pct start 106
__sync_wait: 36 An error occurred in another process (expected sequence number 7)
__lxc_start: 1999 Failed to spawn container "106"
startup for container '106' failed
root@proxmox:~#



root@proxmox:~# pveversion -v
proxmox-ve: 6.4-1 (running kernel: 5.11.17-1-pve)
pve-manager: 6.4-8 (running version: 6.4-8/185e14db)
pve-kernel-5.11: 7.0-1~bpo10
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.11.17-1-pve: 5.11.17-1~bpo10
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-4.15: 5.4-19
pve-kernel-4.15.18-30-pve: 4.15.18-58
pve-kernel-4.13.4-1-pve: 4.13.4-26

root@proxmox:~# pct start 110
root@proxmox:~# pct start 101
root@proxmox:~# pct start 106
__sync_wait: 36 An error occurred in another process (expected sequence number 7)
startup for container '106' failed
root@proxmox:~# pct list
VMID Status Lock Name
101 running xxxx
106 stopped xxxxcxx
110 running xxxxxx
root@proxmox:~#

Code:

root@proxmox:~# lxc-start -n 106 -F -l DEBUG -o /tmp/lxc-106.log
lxc-start: 106: sync.c: __sync_wait: 36 An error occurred in another process (expected sequence number 7)
                         lxc-start: 106: start.c: __lxc_start: 1999 Failed to spawn container "106"
                   lxc-start: 106: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: 106: tools/lxc_start.c: main: 314 Additional information can be obtained by setting the --logfile and --logpriority options
root@proxmox:~#

/tmp/lxc-106.log

Code:

lxc-start 106 20210604160243.670 INFO     lsm - lsm/lsm.c:lsm_init:40 - Initialized LSM security driver AppArmor
lxc-start 106 20210604160243.670 INFO     conf - conf.c:run_script_argv:331 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "106", config section$
lxc-start 106 20210604160244.720 DEBUG    terminal - terminal.c:lxc_terminal_peer_default:672 - Using terminal "/dev/tty" as proxy
lxc-start 106 20210604160244.720 DEBUG    terminal - terminal.c:lxc_terminal_winsz:61 - Set window size to 100 columns and 31 rows
lxc-start 106 20210604160244.721 DEBUG    seccomp - seccomp.c:parse_config_v2:656 - Host native arch is [3221225534]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "reject_force_umount  # comment this to allow umount -f;  not recommended"
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:524 - Set seccomp rule to reject force umounts
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "[all]"
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "kexec_load errno 1"
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[246:kexec_load] action[327681:errno] arch[0]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[246:kexec_load] action[327681:errno] arch[1073741827]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[246:kexec_load] action[327681:errno] arch[1073741886]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "open_by_handle_at errno 1"
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[304:open_by_handle_at] action[327681:errno] arch[0]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[304:open_by_handle_at] action[327681:errno] arch[1073741$
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[304:open_by_handle_at] action[327681:errno] arch[1073741$
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "init_module errno 1"
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[175:init_module] action[327681:errno] arch[0]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[175:init_module] action[327681:errno] arch[1073741827]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[175:init_module] action[327681:errno] arch[1073741886]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "finit_module errno 1"
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding native rule for syscall[313:finit_module] action[327681:errno] arch[0]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[313:finit_module] action[327681:errno] arch[1073741827]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:do_resolve_add_rule:566 - Adding compat rule for syscall[313:finit_module] action[327681:errno] arch[1073741886]
lxc-start 106 20210604160244.721 INFO     seccomp - seccomp.c:parse_config_v2:807 - Processing "delete_module errno 1"
lxc-start 106 20210604160246.354 ERROR    sync - sync.c:__sync_wait:36 - An error occurred in another process (expected sequence number 7)
lxc-start 106 20210604160246.354 DEBUG    network - network.c:lxc_delete_network:3672 - Deleted network devices
lxc-start 106 20210604160246.354 ERROR    start - start.c:__lxc_start:1999 - Failed to spawn container "106"
lxc-start 106 20210604160246.354 WARN     start - start.c:lxc_abort:1013 - No such process - Failed to send SIGKILL via pidfd 55 for process 8429
lxc-start 106 20210604160246.910 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/unified//lxc.monitor/106/lxc.pivot"
lxc-start 106 20210604160246.910 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/unified//lxc.monitor/106"
lxc-start 106 20210604160246.910 WARN     cgfsng - cgroups/cgfsng.c:cgfsng_monitor_destroy:1103 - Failed to destroy "/sys/fs/cgroup/unified//lxc.monitor/106"
lxc-start 106 20210604160246.910 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/systemd//lxc.monitor/106/lxc.pivot"
lxc-start 106 20210604160246.910 WARN
lxc-start 106 20210604160246.930 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/blkio//lxc.monitor/106/lxc.pivot"
lxc-start 106 20210604160246.930 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/blkio//lxc.monitor/106"
lxc-start 106 20210604160246.930 WARN     cgfsng - cgroups/cgfsng.c:cgfsng_monitor_destroy:1103 - Failed to destroy "/sys/fs/cgroup/blkio//lxc.monitor/106"
lxc-start 106 20210604160246.931 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/memory//lxc.monitor/106/lxc.pivot"
lxc-start 106 20210604160246.931 WARN     utils - utils.c:lxc_rm_rf:1895 - Device or resource busy - Failed to delete "/sys/fs/cgroup/memory//lxc.monitor/106"
lxc-start 106 20210604160246.931 WARN     cgfsng - cgroups/cgfsng.c:cgfsng_monitor_destroy:1103 - Failed to destroy "/sys/fs/cgroup/memory//lxc.monitor/106"
lxc-start 106 20210604160246.931 INFO     conf - conf.c:run_script_argv:331 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "106", config section "lxc"
lxc-start 106 20210604160247.434 INFO     conf - conf.c:run_script_argv:331 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "106", config section$
lxc-start 106 20210604160248.508 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start 106 20210604160248.508 ERROR    lxc_start - tools/lxc_start.c:main:314 - Additional information can be obtained by setting the --logfile and --logpriority options

Stoiko Ivanov · Jun 7, 2021

Could you please post the containers configuration? `pct config 106`

notter · Jun 8, 2021

Stoiko Ivanov said:
Could you please post the containers configuration? `pct config 106`

root@proxmox:~# pct config 106
arch: amd64
cores: 8
hostname: server.XXXXXX
lock: mounted
memory: 16000
nameserver: 195.175.39.39 195.175.39.40
net0: name=eth0,bridge=vmbr0,gw=X.X.X.X,hwaddr=56:53:C8:09:3F:E5,ip=X.X.X.X/32,type=veth
ostype: centos
rootfs: local-lvm:vm-106-disk-1,quota=1,size=450G
searchdomain: XXXXXX
swap: 4096
root@proxmox:~#

notter · Jun 11, 2021

do you have anything to suggest?

Search

Search

container is not running.

notter

Member

Stoiko Ivanov

Proxmox Staff Member

notter

Member

notter

Member

Stoiko Ivanov

Proxmox Staff Member

notter

Member

Stoiko Ivanov

Proxmox Staff Member

notter

Member

Stoiko Ivanov

Proxmox Staff Member

notter

Member

notter

Member