[SOLVED] Container not start after proxmox upgrade

Euphoria · Aug 24, 2018

Hi to the community, this is my first post. I have a proxmox suscription and yesterday made a upgrade to proxmox. after a proxmox reboot (because kernel upgrade) the lxc dont start again

the only question ask me when i upgrade is because im setup nfs share for container and add 2 line lxc-default-cgns.
Here some info:
Error from proxmox web gui:

Code:

Job for pve-container@100.service failed because the control process exited with error code.
See "systemctl status pve-container@100.service" and "journalctl -xe" for details.
TASK ERROR: command 'systemctl start pve-container@100' failed: exit code 1

pveversion -v

Code:

root@pve:/etc/apparmor.d/lxc# pveversion -v
proxmox-ve: 5.2-2 (running kernel: 4.15.18-2-pve)
pve-manager: 5.2-7 (running version: 5.2-7/8d88e66a)
pve-kernel-4.15: 5.2-5
pve-kernel-4.13: 5.2-2
pve-kernel-4.15.18-2-pve: 4.15.18-20
pve-kernel-4.15.18-1-pve: 4.15.18-19
pve-kernel-4.15.17-3-pve: 4.15.17-14
pve-kernel-4.15.17-2-pve: 4.15.17-10
pve-kernel-4.15.17-1-pve: 4.15.17-9
pve-kernel-4.13.16-4-pve: 4.13.16-51
pve-kernel-4.13.16-3-pve: 4.13.16-50
pve-kernel-4.13.16-2-pve: 4.13.16-48
pve-kernel-4.13.16-1-pve: 4.13.16-46
pve-kernel-4.13.13-6-pve: 4.13.13-42
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-2-pve: 4.13.13-33
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-38
libpve-guest-common-perl: 2.0-17
libpve-http-server-perl: 2.0-10
libpve-storage-perl: 5.0-24
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.2+pve1-1
lxcfs: 3.0.0-1
novnc-pve: 1.0.0-2
proxmox-widget-toolkit: 1.0-19
pve-cluster: 5.0-29
pve-container: 2.0-25
pve-docs: 5.2-8
pve-firewall: 3.0-13
pve-firmware: 2.0-5
pve-ha-manager: 2.0-5
pve-i18n: 1.0-6
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.2-1
pve-xtermjs: 1.0-5
qemu-server: 5.0-32
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.9-pve1~bpo9
root@pve:/etc/apparmor.d/lxc#

apparmor_parser -r -W -T /etc/apparmor.d/lxc-containers (find when search for the error)

Code:

root@pve:/etc/apparmor.d/lxc# apparmor_parser -r -W -T /etc/apparmor.d/lxc-containers
Multiple definitions for profile lxc-container-default-cgns exist,bailing out.
root@pve:/etc/apparmor.d/lxc#

nano lxc-default-cgns (the only diference y cgroup2 but no idea whats is this.)

Code:

# Do not load this file.  Rather, load /etc/apparmor.d/lxc-containers, which
# will source all profiles under /etc/apparmor.d/lxc

profile lxc-container-default-cgns flags=(attach_disconnected,mediate_deleted) {
  #include <abstractions/lxc/container-base>

  # the container may never be allowed to mount devpts.  If it does, it
  # will remount the host's devpts.  We could allow it to do it with
  # the newinstance option (but, right now, we don't).
  deny mount fstype=devpts,
  mount fstype=cgroup -> /sys/fs/cgroup/**,
  mount fstype=cgroup2 -> /sys/fs/cgroup/**,
  mount fstype=nfs*,
  mount fstype=rpc_pipefs,
}

lxc-start -F -n 100 --logfile=lxc.log --logpriority=debug

Code:

lxc-start 100 20180824093042.838 INFO     lsm - lsm/lsm.c:lsm_init:47 - LSM security driver AppArmor
lxc-start 100 20180824093042.840 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "reject_force_umount  # comment this to allow umount -f;  not recommended"
lxc-start 100 20180824093042.840 INFO     seccomp - seccomp.c:do_resolve_add_rule:503 - Set seccomp rule to reject force umounts
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:parse_config_v2:934 - Added native rule for arch 0 for reject_force_umount action 0(kill)
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:do_resolve_add_rule:503 - Set seccomp rule to reject force umounts
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:parse_config_v2:943 - Added compat rule for arch 1073741827 for reject_force_umount action 0(kill)
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:do_resolve_add_rule:503 - Set seccomp rule to reject force umounts
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:parse_config_v2:953 - Added compat rule for arch 1073741886 for reject_force_umount action 0(kill)
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:do_resolve_add_rule:503 - Set seccomp rule to reject force umounts
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:parse_config_v2:963 - Added native rule for arch -1073741762 for reject_force_umount action 0(kill)
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "[all]"
lxc-start 100 20180824093042.841 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "kexec_load errno 1"
lxc-start 100 20180824093042.842 INFO     seccomp - seccomp.c:parse_config_v2:934 - Added native rule for arch 0 for kexec_load action 327681(errno)
lxc-start 100 20180824093042.842 INFO     seccomp - seccomp.c:parse_config_v2:943 - Added compat rule for arch 1073741827 for kexec_load action 327681(errno)
lxc-start 100 20180824093042.842 INFO     seccomp - seccomp.c:parse_config_v2:953 - Added compat rule for arch 1073741886 for kexec_load action 327681(errno)
lxc-start 100 20180824093042.842 INFO     seccomp - seccomp.c:parse_config_v2:963 - Added native rule for arch -1073741762 for kexec_load action 327681(errno)
lxc-start 100 20180824093042.842 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "open_by_handle_at errno 1"
lxc-start 100 20180824093042.842 INFO     seccomp - seccomp.c:parse_config_v2:934 - Added native rule for arch 0 for open_by_handle_at action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:943 - Added compat rule for arch 1073741827 for open_by_handle_at action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:953 - Added compat rule for arch 1073741886 for open_by_handle_at action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:963 - Added native rule for arch -1073741762 for open_by_handle_at action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "init_module errno 1"
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:934 - Added native rule for arch 0 for init_module action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:943 - Added compat rule for arch 1073741827 for init_module action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:953 - Added compat rule for arch 1073741886 for init_module action 327681(errno)
lxc-start 100 20180824093042.843 INFO     seccomp - seccomp.c:parse_config_v2:963 - Added native rule for arch -1073741762 for init_module action 327681(errno)
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "finit_module errno 1"
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:934 - Added native rule for arch 0 for finit_module action 327681(errno)
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:943 - Added compat rule for arch 1073741827 for finit_module action 327681(errno)
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:953 - Added compat rule for arch 1073741886 for finit_module action 327681(errno)
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:963 - Added native rule for arch -1073741762 for finit_module action 327681(errno)
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:757 - Processing "delete_module errno 1"
lxc-start 100 20180824093042.844 INFO     seccomp - seccomp.c:parse_config_v2:934 - Added native rule for arch 0 for delete_module action 327681(errno)
lxc-start 100 20180824093042.845 INFO     seccomp - seccomp.c:parse_config_v2:943 - Added compat rule for arch 1073741827 for delete_module action 327681(errno)
lxc-start 100 20180824093042.845 INFO     seccomp - seccomp.c:parse_config_v2:953 - Added compat rule for arch 1073741886 for delete_module action 327681(errno)
lxc-start 100 20180824093042.845 INFO     seccomp - seccomp.c:parse_config_v2:963 - Added native rule for arch -1073741762 for delete_module action 327681(errno)
lxc-start 100 20180824093042.845 INFO     seccomp - seccomp.c:parse_config_v2:967 - Merging compat seccomp contexts into main context
lxc-start 100 20180824093042.846 INFO     conf - conf.c:run_script_argv:374 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "100", config section "lxc"
lxc-start 100 20180824093043.624 DEBUG    terminal - terminal.c:lxc_terminal_peer_default:711 - Using terminal "/dev/tty" as proxy
lxc-start 100 20180824093043.624 DEBUG    terminal - terminal.c:lxc_terminal_signal_init:189 - Created signal fd 9
lxc-start 100 20180824093043.624 DEBUG    terminal - terminal.c:lxc_terminal_winsz:87 - Set window size to 106 columns and 23 rows
lxc-start 100 20180824093043.625 INFO     start - start.c:lxc_init:873 - Container "100" is initialized
lxc-start 100 20180824093043.627 INFO     conf - conf.c:run_script:512 - Executing script "/usr/share/lxc/lxcnetaddbr" for container "100", config section "net"
lxc-start 100 20180824093044.647 DEBUG    network - network.c:instantiate_veth:205 - Instantiated veth "veth100i0/vethO36H3I", index is "15"
lxc-start 100 20180824093044.647 DEBUG    cgfsng - cgroups/cgfsng.c:cg_legacy_handle_cpuset_hierarchy:613 - "cgroup.clone_children" was already set to "1"
lxc-start 100 20180824093044.647 ERROR    cgfsng - cgroups/cgfsng.c:create_path_for_hierarchy:1211 - The cgroup "/sys/fs/cgroup/rdma//lxc/100" already existed
lxc-start 100 20180824093044.647 ERROR    cgfsng - cgroups/cgfsng.c:cgfsng_create:1325 - Failed to create cgroup "/sys/fs/cgroup/rdma//lxc/100"
lxc-start 100 20180824093044.647 DEBUG    cgfsng - cgroups/cgfsng.c:cg_legacy_handle_cpuset_hierarchy:613 - "cgroup.clone_children" was already set to "1"
lxc-start 100 20180824093044.649 INFO     start - start.c:lxc_spawn:1678 - Cloned CLONE_NEWNS
lxc-start 100 20180824093044.649 INFO     start - start.c:lxc_spawn:1678 - Cloned CLONE_NEWPID
lxc-start 100 20180824093044.649 INFO     start - start.c:lxc_spawn:1678 - Cloned CLONE_NEWUTS
lxc-start 100 20180824093044.649 INFO     start - start.c:lxc_spawn:1678 - Cloned CLONE_NEWIPC
lxc-start 100 20180824093044.649 INFO     start - start.c:lxc_spawn:1678 - Cloned CLONE_NEWNET
lxc-start 100 20180824093044.649 DEBUG    start - start.c:lxc_try_preserve_namespaces:205 - Preserved mnt namespace via fd 15
lxc-start 100 20180824093044.649 DEBUG    start - start.c:lxc_try_preserve_namespaces:205 - Preserved pid namespace via fd 16
lxc-start 100 20180824093044.649 DEBUG    start - start.c:lxc_try_preserve_namespaces:205 - Preserved uts namespace via fd 17
lxc-start 100 20180824093044.649 DEBUG    start - start.c:lxc_try_preserve_namespaces:205 - Preserved ipc namespace via fd 18
lxc-start 100 20180824093044.649 DEBUG    start - start.c:lxc_try_preserve_namespaces:205 - Preserved net namespace via fd 19
lxc-start 100 20180824093044.649 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "memory.limit_in_bytes" set to "1073741824"
lxc-start 100 20180824093044.649 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "memory.memsw.limit_in_bytes" set to "1610612736"
lxc-start 100 20180824093044.649 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "cpu.shares" set to "1024"
lxc-start 100 20180824093044.649 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "cpuset.cpus" set to "0,6"
lxc-start 100 20180824093044.649 INFO     cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2256 - Limits for the legacy cgroup hierarchies have been setup
lxc-start 100 20180824093044.650 DEBUG    start - start.c:lxc_spawn:1732 - Preserved net namespace via fd 10
lxc-start 100 20180824093044.671 DEBUG    network - network.c:lxc_network_move_created_netdev_priv:2489 - Moved network device "vethO36H3I"/"eth0" to network namespace of 5454
lxc-start 100 20180824093044.671 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.deny" set to "a"
lxc-start 100 20180824093044.671 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c *:* m"
lxc-start 100 20180824093044.671 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "b *:* m"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 1:3 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 1:5 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 1:7 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 5:0 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 5:1 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 5:2 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 1:8 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 1:9 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 136:* rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 10:229 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 254:0 rm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 10:200 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 10:228 rwm"
lxc-start 100 20180824093044.672 DEBUG    cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2251 - Set controller "devices.allow" set to "c 10:232 rwm"
lxc-start 100 20180824093044.672 INFO     cgfsng - cgroups/cgfsng.c:__cg_legacy_setup_limits:2256 - Limits for the legacy cgroup hierarchies have been setup
lxc-start 100 20180824093044.673 INFO     start - start.c:do_start:1225 - Unshared CLONE_NEWCGROUP
lxc-start 100 20180824093044.673 DEBUG    storage - storage/storage.c:storage_query:251 - Detected rootfs type "dir"
lxc-start 100 20180824093044.673 DEBUG    conf - conf.c:lxc_mount_rootfs:1343 - Mounted rootfs "/var/lib/lxc/100/rootfs" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs" with options "(null)"
lxc-start 100 20180824093044.673 INFO     conf - conf.c:setup_utsname:802 - Set hostname to "plex"
lxc-start 100 20180824093044.691 DEBUG    network - network.c:setup_hw_addr:2756 - Mac address "6E:C2:46:5D:12:CA" on "eth0" has been setup
lxc-start 100 20180824093044.691 DEBUG    network - network.c:lxc_setup_netdev_in_child_namespaces:3021 - Network device "eth0" has been setup
lxc-start 100 20180824093044.691 INFO     network - network.c:lxc_setup_network_in_child_namespaces:3042 - network has been setup
lxc-start 100 20180824093044.691 INFO     conf - conf.c:mount_autodev:1129 - Preparing "/dev"
lxc-start 100 20180824093044.691 INFO     conf - conf.c:mount_autodev:1176 - Prepared "/dev"
lxc-start 100 20180824093044.698 INFO     conf - conf.c:run_script_argv:374 - Executing script "/usr/share/lxcfs/lxc.mount.hook" for container "100", config section "lxc"
lxc-start 100 20180824093044.700 DEBUG    conf - conf.c:run_buffer:344 - Script exec /usr/share/lxcfs/lxc.mount.hook 100 lxc mount with output: missing /var/lib/lxcfs/proc/ - lxcfs not running?

lxc-start 100 20180824093044.700 ERROR    conf - conf.c:run_buffer:353 - Script exited with status 1
lxc-start 100 20180824093044.700 ERROR    conf - conf.c:lxc_setup:3626 - Failed to run mount hooks
lxc-start 100 20180824093044.700 ERROR    start - start.c:do_start:1246 - Failed to setup container "100"
lxc-start 100 20180824093044.700 ERROR    sync - sync.c:__sync_wait:59 - An error occurred in another process (expected sequence number 5)
lxc-start 100 20180824093044.701 INFO     network - network.c:lxc_delete_network_priv:2573 - Interface "eth0" with index 15 already deleted or existing in different network namespace
lxc-start 100 20180824093044.701 INFO     network - network.c:lxc_delete_network_priv:2583 - Removed interface "eth0" with index 15
lxc-start 100 20180824093044.759 INFO     network - network.c:lxc_delete_network_priv:2605 - Removed interface "veth100i0" from ""
lxc-start 100 20180824093044.759 DEBUG    network - network.c:lxc_delete_network:3169 - Deleted network devices
lxc-start 100 20180824093044.759 ERROR    start - start.c:__lxc_start:1948 - Failed to spawn container "100"
lxc-start 100 20180824093044.761 INFO     conf - conf.c:run_script_argv:374 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "100", config section "lxc"
lxc-start 100 20180824093045.264 INFO     conf - conf.c:run_script_argv:374 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "100", config section "lxc"
lxc-start 100 20180824093046.415 ERROR    lxc_start - tools/lxc_start.c:main:330 - The container failed to start
lxc-start 100 20180824093046.415 ERROR    lxc_start - tools/lxc_start.c:main:336 - Additional information can be obtained by setting the --logfile and --logpriority options

Apt log (sorry for some bad character, the vnc web console do it if you press some key)

Code:

Log started: 2018-08-12  21:58:10
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 102174 files and directories currently installed.)
Preparing to unpack .../linux-libc-dev_4.9.110-3+deb9u1_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.9.110-3+deb9u1) over (4.9.110-1) ...
Setting up linux-libc-dev:amd64 (4.9.110-3+deb9u1) ...
Log ended: 2018-08-12  21:58:14

Log started: 2018-08-23  22:57:59
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 102174 files and directories currently installed.)
Preparing to unpack .../00-libsmbclient_2%3a4.5.12+dfsg-2+deb9u3_amd64.deb ...
Unpacking libsmbclient:amd64 (2:4.5.12+dfsg-2+deb9u3) over (2:4.5.12+dfsg-2+deb9u2) ...
Preparing to unpack .../01-samba-libs_2%3a4.5.12+dfsg-2+deb9u3_amd64.deb ...
Unpacking samba-libs:amd64 (2:4.5.12+dfsg-2+deb9u3) over (2:4.5.12+dfsg-2+deb9u2) ...
Preparing to unpack .../02-libwbclient0_2%3a4.5.12+dfsg-2+deb9u3_amd64.deb ...
Unpacking libwbclient0:amd64 (2:4.5.12+dfsg-2+deb9u3) over (2:4.5.12+dfsg-2+deb9u2) ...
Preparing to unpack .../03-smbclient_2%3a4.5.12+dfsg-2+deb9u3_amd64.deb ...
Unpacking smbclient (2:4.5.12+dfsg-2+deb9u3) over (2:4.5.12+dfsg-2+deb9u2) ...
Preparing to unpack .../04-samba-common_2%3a4.5.12+dfsg-2+deb9u3_all.deb ...
Unpacking samba-common (2:4.5.12+dfsg-2+deb9u3) over (2:4.5.12+dfsg-2+deb9u2) ...
Preparing to unpack .../05-openssh-sftp-server_1%3a7.4p1-10+deb9u4_amd64.deb ...
Unpacking openssh-sftp-server (1:7.4p1-10+deb9u4) over (1:7.4p1-10+deb9u3) ...
Preparing to unpack .../06-openssh-server_1%3a7.4p1-10+deb9u4_amd64.deb ...
Unpacking openssh-server (1:7.4p1-10+deb9u4) over (1:7.4p1-10+deb9u3) ...
Preparing to unpack .../07-openssh-client_1%3a7.4p1-10+deb9u4_amd64.deb ...
Unpacking openssh-client (1:7.4p1-10+deb9u4) over (1:7.4p1-10+deb9u3) ...
Preparing to unpack .../08-ssh_1%3a7.4p1-10+deb9u4_all.deb ...
Unpacking ssh (1:7.4p1-10+deb9u4) over (1:7.4p1-10+deb9u3) ...
Preparing to unpack .../09-libpve-common-perl_5.0-38_all.deb ...
Unpacking libpve-common-perl (5.0-38) over (5.0-35) ...
Preparing to unpack .../10-libpve-http-server-perl_2.0-10_all.deb ...
Unpacking libpve-http-server-perl (2.0-10) over (2.0-9) ...
Preparing to unpack .../11-linux-libc-dev_4.9.110-3+deb9u4_amd64.deb ...
Unpacking linux-libc-dev:amd64 (4.9.110-3+deb9u4) over (4.9.110-3+deb9u1) ...
Preparing to unpack .../12-lxc-pve_3.0.2+pve1-1_amd64.deb ...
Unpacking lxc-pve (3.0.2+pve1-1) over (3.0.0-3) ...
Preparing to unpack .../13-novnc-pve_1.0.0-2_amd64.deb ...
Unpacking novnc-pve (1.0.0-2) over (1.0.0-1) ...
Preparing to unpack .../14-pve-cluster_5.0-29_amd64.deb ...
Unpacking pve-cluster (5.0-29) over (5.0-28) ...
Preparing to unpack .../15-pve-container_2.0-25_all.deb ...
Unpacking pve-container (2.0-25) over (2.0-24) ...
Selecting previously unselected package pve-kernel-4.15.18-2-pve.
Preparing to unpack .../16-pve-kernel-4.15.18-2-pve_4.15.18-20_amd64.deb ...
Unpacking pve-kernel-4.15.18-2-pve (4.15.18-20) ...
Preparing to unpack .../17-pve-kernel-4.15_5.2-5_all.deb ...
Unpacking pve-kernel-4.15 (5.2-5) over (5.2-4) ...
Preparing to unpack .../18-pve-kernel-4.15.18-1-pve_4.15.18-19_amd64.deb ...
Unpacking pve-kernel-4.15.18-1-pve (4.15.18-19) over (4.15.18-15) ...
Preparing to unpack .../19-pve-docs_5.2-8_all.deb ...
Unpacking pve-docs (5.2-8) over (5.2-4) ...
Preparing to unpack .../20-qemu-server_5.0-32_amd64.deb ...
Unpacking qemu-server (5.0-32) over (5.0-29) ...
Preparing to unpack .../21-pve-manager_5.2-7_amd64.deb ...
Unpacking pve-manager (5.2-7) over (5.2-5) ...
Setting up lxc-pve (3.0.2+pve1-1) ...
Installing new version of config file /etc/apparmor.d/abstractions/lxc/start-container ...

Configuration file '/etc/apparmor.d/lxc/lxc-default-cgns'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** lxc-default-cgns (Y/I/N/O/D/Z) [default=N] ? d
[?1049h[?1h=
--- /etc/apparmor.d/lxc/lxc-default-cgns        2018-06-10 14:39:38.359687695 +0100
+++ /etc/apparmor.d/lxc/lxc-default-cgns.dpkg-new       2018-08-20 10:51:27.000000000 +0100
@@ -9,6 +9,5 @@
   # the newinstance option (but, right now, we don't).
   deny mount fstype=devpts,
   mount fstype=cgroup -> /sys/fs/cgroup/**,
-  mount fstype=nfs*,
-  mount fstype=rpc_pipefs,
+  mount fstype=cgroup2 -> /sys/fs/cgroup/**,
 }
[7m(END)[27m[K
[K[?1l>[?1049l
Configuration file '/etc/apparmor.d/lxc/lxc-default-cgns'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** lxc-default-cgns (Y/I/N/O/D/Z) [default=N] ? y^H^H^H^H
Installing new version of config file /etc/apparmor.d/lxc/lxc-default-cgns ...
Installing new version of config file /etc/apparmor.d/lxc/lxc-default-with-nesting ...
Multiple definitions for profile lxc-container-default-cgns exist,bailing out.
Setting up libwbclient0:amd64 (2:4.5.12+dfsg-2+deb9u3) ...
Setting up samba-libs:amd64 (2:4.5.12+dfsg-2+deb9u3) ...
Setting up libpve-common-perl (5.0-38) ...
Setting up linux-libc-dev:amd64 (4.9.110-3+deb9u4) ...
Setting up samba-common (2:4.5.12+dfsg-2+deb9u3) ...
Setting up libsmbclient:amd64 (2:4.5.12+dfsg-2+deb9u3) ...
Setting up pve-docs (5.2-8) ...
Setting up smbclient (2:4.5.12+dfsg-2+deb9u3) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Setting up novnc-pve (1.0.0-2) ...
Processing triggers for systemd (232-25+deb9u4) ...
Setting up pve-kernel-4.15.18-1-pve (4.15.18-19) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.15.18-1-pve /boot/vmlinuz-4.15.18-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.15.18-1-pve /boot/vmlinuz-4.15.18-1-pve
update-initramfs: Generating /boot/initrd.img-4.15.18-1-pve
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.15.18-1-pve /boot/vmlinuz-4.15.18-1-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.15.18-2-pve
Found linux image: /boot/vmlinuz-4.15.18-1-pve
Found initrd image: /boot/initrd.img-4.15.18-1-pve
Found linux image: /boot/vmlinuz-4.15.17-3-pve
Found initrd image: /boot/initrd.img-4.15.17-3-pve
Found linux image: /boot/vmlinuz-4.15.17-2-pve
Found initrd image: /boot/initrd.img-4.15.17-2-pve
Found linux image: /boot/vmlinuz-4.15.17-1-pve
Found initrd image: /boot/initrd.img-4.15.17-1-pve
Found linux image: /boot/vmlinuz-4.13.16-4-pve
Found initrd image: /boot/initrd.img-4.13.16-4-pve
Found linux image: /boot/vmlinuz-4.13.16-3-pve
Found initrd image: /boot/initrd.img-4.13.16-3-pve
Found linux image: /boot/vmlinuz-4.13.16-2-pve
Found initrd image: /boot/initrd.img-4.13.16-2-pve
Found linux image: /boot/vmlinuz-4.13.16-1-pve
Found initrd image: /boot/initrd.img-4.13.16-1-pve
Found linux image: /boot/vmlinuz-4.13.13-6-pve
Found initrd image: /boot/initrd.img-4.13.13-6-pve
Found linux image: /boot/vmlinuz-4.13.13-5-pve
Found initrd image: /boot/initrd.img-4.13.13-5-pve
Found linux image: /boot/vmlinuz-4.13.13-2-pve
Found initrd image: /boot/initrd.img-4.13.13-2-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
done
Processing triggers for man-db (2.7.6.1-2) ...
Setting up pve-kernel-4.15.18-2-pve (4.15.18-20) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.15.18-2-pve /boot/vmlinuz-4.15.18-2-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.15.18-2-pve /boot/vmlinuz-4.15.18-2-pve
update-initramfs: Generating /boot/initrd.img-4.15.18-2-pve
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.15.18-2-pve /boot/vmlinuz-4.15.18-2-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.15.18-2-pve
Found initrd image: /boot/initrd.img-4.15.18-2-pve
Found linux image: /boot/vmlinuz-4.15.18-1-pve
Found initrd image: /boot/initrd.img-4.15.18-1-pve
Found linux image: /boot/vmlinuz-4.15.17-3-pve
Found initrd image: /boot/initrd.img-4.15.17-3-pve
Found linux image: /boot/vmlinuz-4.15.17-2-pve
Found initrd image: /boot/initrd.img-4.15.17-2-pve
Found linux image: /boot/vmlinuz-4.15.17-1-pve
Found initrd image: /boot/initrd.img-4.15.17-1-pve
Found linux image: /boot/vmlinuz-4.13.16-4-pve
Found initrd image: /boot/initrd.img-4.13.16-4-pve
Found linux image: /boot/vmlinuz-4.13.16-3-pve
Found initrd image: /boot/initrd.img-4.13.16-3-pve
Found linux image: /boot/vmlinuz-4.13.16-2-pve
Found initrd image: /boot/initrd.img-4.13.16-2-pve
Found linux image: /boot/vmlinuz-4.13.16-1-pve
Found initrd image: /boot/initrd.img-4.13.16-1-pve
Found linux image: /boot/vmlinuz-4.13.13-6-pve
Found initrd image: /boot/initrd.img-4.13.13-6-pve
Found linux image: /boot/vmlinuz-4.13.13-5-pve
Found initrd image: /boot/initrd.img-4.13.13-5-pve
Found linux image: /boot/vmlinuz-4.13.13-2-pve
Found initrd image: /boot/initrd.img-4.13.13-2-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
done
Setting up pve-cluster (5.0-29) ...
Setting up pve-kernel-4.15 (5.2-5) ...
Setting up openssh-client (1:7.4p1-10+deb9u4) ...
Setting up libpve-http-server-perl (2.0-10) ...
Setting up qemu-server (5.0-32) ...
Setting up openssh-sftp-server (1:7.4p1-10+deb9u4) ...
Setting up pve-container (2.0-25) ...
Processing triggers for pve-ha-manager (2.0-5) ...
Setting up pve-manager (5.2-7) ...
Setting up openssh-server (1:7.4p1-10+deb9u4) ...
Setting up ssh (1:7.4p1-10+deb9u4) ...
Processing triggers for systemd (232-25+deb9u4) ...
Log ended: 2018-08-23  23:05:31

Log started: 2018-08-24  10:24:40
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 108399 files and directories currently installed.)
Preparing to unpack .../lxc-pve_3.0.2+pve1-1_amd64.deb ...
Unpacking lxc-pve (3.0.2+pve1-1) over (3.0.2+pve1-1) ...
Setting up lxc-pve (3.0.2+pve1-1) ...
Multiple definitions for profile lxc-container-default-cgns exist,bailing out.
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for man-db (2.7.6.1-2) ...
Log ended: 2018-08-24  10:24:48

i made a upgrade some week before and all work fine!

Any help are appreciated!

Thx. in advance.

sahostking · Aug 24, 2018

Not sure. But it may be related to the same issue I had awhile ago:
https://forum.proxmox.com/threads/m...iled-read-only-file-system.45080/#post-215115

Euphoria · Aug 24, 2018

sahostking said:
Not sure. But it may be related to the same issue I had awhile ago:
https://forum.proxmox.com/threads/m...iled-read-only-file-system.45080/#post-215115

Hi, thank for help.

I made a new lxc container and try it but not work

Code:

find: ‘/sys/fs/cgroup/*/lxc/104*’: No such file or directory

I thing is something wrong with apparmor but not sure.

sahostking · Aug 24, 2018

when you start the container what does "journalctl -xe" provide you with. It will usually tell you what the "real" issue is.

Stoiko Ivanov · Aug 24, 2018

Euphoria said:
lxc-start 100 20180824093044.698 INFO conf - conf.c:run_script_argv:374 - Executing script "/usr/share/lxcfs/lxc.mount.hook" for container "100", config section "lxc" lxc-start 100 20180824093044.700 DEBUG conf - conf.c:run_buffer:344 - Script exec /usr/share/lxcfs/lxc.mount.hook 100 lxc mount with output: missing /var/lib/lxcfs/proc/ - lxcfs not running?

* does `lxcfs` run? (systemctl -a |grep lxcfs/ systemctl status -l lxcfs.service)

Euphoria said:
root@pve:/etc/apparmor.d/lxc# apparmor_parser -r -W -T /etc/apparmor.d/lxc-containers Multiple definitions for profile lxc-container-default-cgns exist,bailing out.

* what's the output of `fgrep -r cgns /etc/apparmor.d` ?

Euphoria · Aug 24, 2018

sahostking said:
when you start the container what does "journalctl -xe" provide you with. It will usually tell you what the "real" issue is.

journalctl -xe (change debian url to XXX because cannot post)

Code:

root@pve:~# journalctl -xe
-- The start-up result is done.
Aug 24 11:34:00 pve systemd[1]: Starting Proxmox VE replication runner...
-- Subject: Unit pvesr.service has begun start-up
-- Defined-By: systemd
-- Support:XXX
--
-- Unit pvesr.service has begun starting up.
Aug 24 11:34:01 pve systemd[1]: Started Proxmox VE replication runner.
-- Subject: Unit pvesr.service has finished start-up
-- Defined-By: systemd
-- Support: XXX
--
-- Unit pvesr.service has finished starting up.
--
-- The start-up result is done.
Aug 24 11:35:00 pve systemd[1]: Starting Proxmox VE replication runner...
-- Subject: Unit pvesr.service has begun start-up
-- Defined-By: systemd
-- Support: XXX
--
-- Unit pvesr.service has begun starting up.
Aug 24 11:35:01 pve systemd[1]: Started Proxmox VE replication runner.
-- Subject: Unit pvesr.service has finished start-up
-- Defined-By: systemd
-- Support: hXXX
--
-- Unit pvesr.service has finished starting up.
--
-- The start-up result is done.
lines 2341-2369/2369 (END)

Stoiko Ivanov said:
* does `lxcfs` run? (systemctl -a |grep lxcfs/ systemctl status -l lxcfs.service)

* what's the output of `fgrep -r cgns /etc/apparmor.d` ?

I restart the server and start again

systemctl -a |grep lxcfs

Code:

root@pve:~# systemctl -a |grep lxcfs
  var-lib-lxcfs.mount                                                                                            loaded    active     mounted   /var/lib/lxcfs                                                                            
  lxcfs.service                                                                                                  loaded    active     running   FUSE filesystem for LXC                                                                    
root@pve:~#

systemctl status -l lxcfs.service

Code:

root@pve:~# systemctl status -l lxcfs.service
● lxcfs.service - FUSE filesystem for LXC
   Loaded: loaded (/lib/systemd/system/lxcfs.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-08-24 11:27:53 WEST; 4min 41s ago
 Main PID: 1136 (lxcfs)
    Tasks: 3 (limit: 7372)
   Memory: 1.0M
      CPU: 10ms
   CGroup: /system.slice/lxcfs.service
           └─1136 /usr/bin/lxcfs /var/lib/lxcfs/

Aug 24 11:27:53 pve lxcfs[1136]:   2: fd:   8: memory
Aug 24 11:27:53 pve lxcfs[1136]:   3: fd:   9: pids
Aug 24 11:27:53 pve lxcfs[1136]:   4: fd:  10: rdma
Aug 24 11:27:53 pve lxcfs[1136]:   5: fd:  11: cpu,cpuacct
Aug 24 11:27:53 pve lxcfs[1136]:   6: fd:  12: hugetlb
Aug 24 11:27:53 pve lxcfs[1136]:   7: fd:  13: blkio
Aug 24 11:27:53 pve lxcfs[1136]:   8: fd:  14: net_cls,net_prio
Aug 24 11:27:53 pve lxcfs[1136]:   9: fd:  15: freezer
Aug 24 11:27:53 pve lxcfs[1136]:  10: fd:  16: devices
Aug 24 11:27:53 pve lxcfs[1136]:  11: fd:  17: name=systemd
root@pve:~#

fgrep -r cgns /etc/apparmor.d

Code:

root@pve:~# fgrep -r cgns /etc/apparmor.d
/etc/apparmor.d/lxc/lxc.log:lxc-start 100 20180824101242.476 ERROR    lsm - lsm/lsm.c:lsm_process_label_set_at:171 - No such file or directory - Failed to set AppArmor label "lxc-container-default-cgns"
/etc/apparmor.d/lxc/lxc.log:lxc-start 100 20180824101242.476 ERROR    apparmor - lsm/apparmor.c:apparmor_process_label_set:1106 - Failed to change AppArmor profile to lxc-container-default-cgns
/etc/apparmor.d/lxc/lxc-default-cgns.dpkg-old:profile lxc-container-default-cgns flags=(attach_disconnected,mediate_deleted) {
/etc/apparmor.d/lxc/lxc-default-cgns.copianfs:profile lxc-container-default-cgns flags=(attach_disconnected,mediate_deleted) {
/etc/apparmor.d/lxc/lxc-default-cgns:profile lxc-container-default-cgns flags=(attach_disconnected,mediate_deleted) {
Binary file /etc/apparmor.d/cache/lxc-containers matches
root@pve:~#

Some error with apparmor label?

Thank you!

Stoiko Ivanov · Aug 24, 2018

try to move the "duplicate" files away from /etc/apparmor.d ( /etc/apparmor.d/lxc/lxc-default-cgns.dpkg-old,
/etc/apparmor.d/lxc/lxc-default-cgns.copianfs ).

Euphoria · Aug 24, 2018

IT WORK!!!

The error is because if you have some other files in /etc/apparmor.d/lxc with bad syntase or bad name, apparmor service dont start.

systemctl start apparmor

Code:

root@pve:~# systemctl start apparmor
Job for apparmor.service failed because the control process exited with error code.
See "systemctl status apparmor.service" and "journalctl -xe" for details.
root@pve:~#

journalctl -xe

Code:

root@pve:~# journalctl -xe
-- Unit apparmor.service has failed.
--
-- The result is failed.
Aug 24 13:47:16 pve systemd[1]: apparmor.service: Unit entered failed state.
Aug 24 13:47:16 pve systemd[1]: apparmor.service: Failed with result 'exit-code'.
Aug 24 13:47:22 pve systemd[1]: Starting AppArmor initialization...
-- Subject: Unit apparmor.service has begun start-up
-- Defined-By: systemd
-- Support: debian website
--
-- Unit apparmor.service has begun starting up.
Aug 24 13:47:23 pve audit[3080]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/usr/bin/lxc-start" pid=3080 comm=
Aug 24 13:47:23 pve kernel: audit: type=1400 audit(1535114843.018:16): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="
Aug 24 13:47:23 pve apparmor[3022]: Starting AppArmor profiles:AppArmor parser error for /etc/apparmor.d/lxc-containers in /etc/apparmor.d/lxc/lxc.log at line 1: Found unexpected charac
Aug 24 13:47:23 pve audit[3090]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/usr/bin/lxc-start" pid=3090 comm=
Aug 24 13:47:23 pve kernel: audit: type=1400 audit(1535114843.030:17): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="
Aug 24 13:47:23 pve apparmor[3022]: AppArmor parser error for /etc/apparmor.d/lxc-containers in /etc/apparmor.d/lxc/lxc.log at line 1: Found unexpected character: '-'
Aug 24 13:47:23 pve apparmor[3022]:  failed!
Aug 24 13:47:23 pve systemd[1]: apparmor.service: Main process exited, code=exited, status=123/n/a
Aug 24 13:47:23 pve systemd[1]: Failed to start AppArmor initialization.
-- Subject: Unit apparmor.service has failed
-- Defined-By: systemd
-- Support: debian website
--
-- Unit apparmor.service has failed.
--
-- The result is failed.
Aug 24 13:47:23 pve systemd[1]: apparmor.service: Unit entered failed state.
Aug 24 13:47:23 pve systemd[1]: apparmor.service: Failed with result 'exit-code'.
root@pve:~#

When the update ask me for replace the lxc-default-cgns im made copy with name "lxc-default-cgns.copianfs" from ssh and also apt create backup "lxc-default-cgns.dpkg-old"

When i made test i create lxc.log file with the command: lxc-start -F -n 100 --logfile=lxc.log --logpriority=debug

I erase the "lxc-default-cgns.copianfs" and "lxc.log" from /etc/apparmor.d/lxc, mv again the "lxc-default-cgns.dpkg-old" to original site (not necessary but do it for chek if this file made also problem) and start apparmor service without errors and lxc work!

Try to restart the server and all the lxc start great.

Thank all for your time and help.

Search

Search

[SOLVED] Container not start after proxmox upgrade

Euphoria

New Member

sahostking

Renowned Member

Euphoria

New Member

sahostking

Renowned Member

Stoiko Ivanov

Proxmox Staff Member

Euphoria

New Member

Stoiko Ivanov

Proxmox Staff Member

Euphoria

New Member