Restarting a CentOS6 container makes host filesystems read-only

[nobody_from_nowhere]

Hi there! There is some old LXC issue https://github.com/lxc/lxd/issues/5486 (https://forum.proxmox.com/threads/problem-with-ct-dont-start-after-shutdown.54782/) , and i know you, guys from Proxmox team, somehow fix it on Proxmox. So, please, can you explain to me, how to do exactly the same on my debian 10? I guess it works somehow with apparmor, but after i have copied /etc/apparmor.d/ - nothing changed. I believe, apparmor should denied remount as we can see on proxmox syslog

Aug  4 23:33:56 test kernel: [11149.862155] audit: type=1400 audit(1596540836.368:77): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/dev/" pid=21831 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.865306] audit: type=1400 audit(1596540836.368:78): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/sys/net/" pid=21833 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.867154] audit: type=1400 audit(1596540836.372:79): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/sys/" pid=21834 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.868787] audit: type=1400 audit(1596540836.372:80): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/sysrq-trigger" pid=21835 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.874872] audit: type=1400 audit(1596540836.380:81): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/sys/devices/virtual/net/" pid=21838 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.876520] audit: type=1400 audit(1596540836.380:82): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/sys/devices/virtual/net/" pid=21839 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.878365] audit: type=1400 audit(1596540836.384:83): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/cpuinfo" pid=21840 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.879960] audit: type=1400 audit(1596540836.384:84): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/diskstats" pid=21841 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.881547] audit: type=1400 audit(1596540836.384:85): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/loadavg" pid=21842 comm="mount" flags="ro, remount"
Aug  4 23:33:56 test kernel: [11149.883390] audit: type=1400 audit(1596540836.388:86): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/proc/meminfo" pid=21843 comm="mount" flags="ro, remount"

but on my clean debian 10 - he doesn't :'(
So, maybe you can prompt to me, how can i do the same?

 

[oguz]

hi,

could you post the output of the following commands:

pveversion -v
pct config CTID

 

[nobody_from_nowhere]

hi there, oguz ! Thx 4 ur attention!

# pveversion -v
proxmox-ve: 6.2-1 (running kernel: 5.4.34-1-pve)
pve-manager: 6.2-4 (running version: 6.2-4/9824574a)
pve-kernel-5.4: 6.2-1
pve-kernel-helper: 6.2-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libproxmox-acme-perl: 1.0.3
libpve-access-control: 6.1-1
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-2
libpve-guest-common-perl: 3.0-10
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve2
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-1
pve-cluster: 6.1-8
pve-container: 3.1-5
pve-docs: 6.2-4
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-2
pve-qemu-kvm: 5.0.0-2
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-2
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1


# pct config 101
arch: amd64
cores: 1
hostname: centos6-1
memory: 254
net0: name=eth0,bridge=vmbr0,hwaddr=0A:F3:F6:65:51:44,type=veth
ostype: centos
rootfs: local-lvm:vm-101-disk-0,size=2G
swap: 254

P.S. I am just want to be clarified: On Proxmox all works correct, i am just want to start centos 6 LXC container on clean debian 10, but i can't because of known issue (see topic). So i thought, maybe anybody from Proxmox team can explain how it has fixed in Proxmox ...
P.S. (2): The same my topic on lxc forum: https://discuss.linuxcontainers.org...ntainer-makes-host-filesystems-read-only/8612

 

[oguz]

P.S. I am just want to be clarified: On Proxmox all works correct, i am just want to start centos 6 LXC container on clean debian 10, but i can't because of known issue (see topic).

oh, okay. but this forum is for proxmox support

i think it might have to do with the fact that you're running the container as privileged. in the github issue it's explained pretty well why this happens [0]

are you also using LVM as the storage backend on your debian 10?


[0]: https://github.com/lxc/lxd/issues/5486#issuecomment-462790776

 

[nobody_from_nowhere]

oh, okay. but this forum is for proxmox support

Yep, i unterstand, but as you can see nobody can help me include LXC team. So i have thought, maybe Proxmox team will want to give me a little hint how to solve the problem, how they have already solved it

i think it might have to do with the fact that you're running the container as privileged. in the github issue it's explained pretty well why this happens [0]

Yeah, ofc i was reading that, but on Proxmox when i create container i also use "privileged mode" and all works correct.

are you also using LVM as the storage backend on your debian 10?


[0]: https://github.com/lxc/lxd/issues/5486#issuecomment-462790776

I have tried lvm and loop, the result the same

 

[oguz]

how exactly are you restarting the container?

 

[nobody_from_nowhere]

how exactly are you restarting the container?

with lxc-start 101 ; lxc-stop 101; lxc-start 101
I think i found the problem. I have copied directory /etc/apparmor.d from Proxmox and then i cleaned all the caches with apparmor_parser --purge-cache or simple rm -rf /var/cache/lxc/apparmor/* /var/cache/apparmor/* . Also i found i use two options: lxc.apparmor.profile = generated and lxc.apparmor.allow_nesting = 1 but it is wrong. They both can not be used at the same time. So, i just removed lxc.apparmor.allow_nesting = 1 and now it seems like all works correct.

Thank you, oguz 4 ur attention and time. You are the only one person who tried to help me, i appreciate that!