1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

lxc containers have extended permissions - acl by default???

Discussion in 'Proxmox VE: Installation and configuration' started by ned, Dec 29, 2015.

  1. ned

    ned Member

    Joined:
    Jan 26, 2015
    Messages:
    75
    Likes Received:
    1
    I am running

    pve-manager/4.1-2/78c5f4a2 (running kernel: 4.2.6-1-pve)

    and have a problem with extended permissions (acls) being set by default on containers. This was not the case before. This might be happening after the upgrade to 4.1, but I am not sure.

    When I create a new container the acls are set but default which makes problems with php-fpm sockets etc.

    Example

    Code:
    root@tester:~# ls -al /var/lib
    total 104
    drwxr-xr-x+ 26 root    root    4096 Jun 30 10:37 .
    drwxr-xr-x+ 11 root    root    4096 Jun 30 10:35 ..
    drwxr-xr-x+  5 root    root    4096 Jun 30 10:36 apt
    drwxr-xr-x+  2 root    root    4096 Nov  8  2014 aptitude
    drwxr-xr-x+  2 root    root    4096 Jun 30 10:37 dbus
    drwxr-xr-x+  2 root    root    4096 Jun 30 10:36 dhcp
    drwxr-xr-x+  3 root    root    4096 Jun 30 10:37 dictionaries-common
    drwxr-xr-x+  7 root    root    4096 Jun 30 10:37 dpkg
    drwxr-xr-x+  2 root    root    4096 Apr  6  2015 initscripts
    drwxr-xr-x+  2 root    root    4096 Apr  6  2015 insserv
    drwxr-xr-x+  3 root    root    4096 Jun 30 10:37 libxml-sax-perl
    drwxr-xr-x+  2 root    root    4096 Nov  8  2014 logrotate
    drwxr-xr-x+  2 root    root    4096 Jun 30 10:36 man-db
    drwxr-xr-x+  2 root    root    4096 May  4  2015 misc
    drwxr-xr-x+  2 root    root    4096 Jun 13  2013 mlocate
    drwxr-xr-x+  2 root    root    4096 Jun 30 10:36 pam
    drwxr-xr-x+  2 postfix postfix 4096 Dec 29 13:32 postfix
    lrwxrwxrwx   1 root    root      18 Jun 30 10:37 python-support -> /usr/lib/pymodules
    drwxr-xr-x+  2 root    root    4096 Jun 30 10:37 sgml-base
    drwxr-xr-x+  3 root    root    4096 Jun 30 10:36 systemd
    drwxr-xr-x+  3 root    root    4096 Jun 30 10:36 tex-common
    drwxr-xr-x+  3 root    root    4096 Jun 30 10:37 ucf
    drwxr-xr-x+  2 root    root    4096 Nov 23  2012 update-rc.d
    drwxr-xr-x+  2 root    root    4096 Apr  6  2015 urandom
    drwxr-xr-x+  3 root    root    4096 Jun 30 10:36 vim
    drwxr-xr-x+  2 root    root    4096 Dec 25  2013 xkb
    drwxr-xr-x+  2 root    root    4096 Jun 30 10:37 xml-core
    

    I can set the cals of with setfacl -b -R /


    How can I turn the acls off by default?
     
    #1 ned, Dec 29, 2015
    Last edited: Dec 29, 2015
  2. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    15,347
    Likes Received:
    171
    What exactly is the problem?
     
  3. ned

    ned Member

    Joined:
    Jan 26, 2015
    Messages:
    75
    Likes Received:
    1
    I get nginx permission denied errors when accessing php-fpm socket in /var/lib/php5-fpm directory. Would like to disable acls by default.
     
  4. windinternet

    windinternet Member

    Joined:
    Oct 8, 2015
    Messages:
    159
    Likes Received:
    5
    Yes, and also, if you put lxc.rootfs.options: noacl (or noatime etc.) in your /etc/pve/lxc/nnn.conf this is not applied.

    The PVE prestart hook for LXC mounts the root without applying the lxc.rootfs.options and with system defaults that also do NOT (probably a jessie bug) honor the default_mount options of the ext4 superblock of the LXC raw image (tried that first). Therefore the acl option is always ON because it is ON for the root of the Proxmox host and that seems to be inherited.

    To fix this, you have to do changes in the perl script in /usr/share/perl5/PVE/LXC.pm. I will try to get the needed changes to the Proxmox team.

    Also in the case of PHP-FPM there is actually a bug in it. It should not fail like this, but it is likely not regularly used on systems with extended acls. I have submitted a pull request with PHP and described it in a bug report https://bugs.php.net/bug.php?id=71223, but that is not a short term solution and there are more programs using sockets that they assume to be group writable after creation without explicitly setting the permission bits, but just using a umask "trick" on creation of the socket file.
     
  5. Skaffen

    Skaffen New Member
    Proxmox VE Subscriber

    Joined:
    Apr 1, 2016
    Messages:
    3
    Likes Received:
    0
    I had an issue with mailman in a Debian Jessie container under PVE 4.1 due to those default posixacls (mailman uses umask rather the specifically setting permissions to set group write on files).

    When digging into the cause of the issue I could see that posixacls are enabled on the root file system for the container, but that still didn't explain why default acls were actually being set on all the directories in the container. The archive for the template doesn't have any extended acls in it all. However it appears that if you use the "--acls" option to gnu tar when extracting a tar archive then even if it doesn't have any default acls in the archive it will set them on directories it creates.

    You can test this by creating a directory with no default acl, creating an archive of it, and then extracting it:
    root@test:~/tartest# mkdir testdir
    root@test:~/tartest# ls -al | grep testdir$
    drwxr-xr-x 2 root root 2 Apr 1 13:59 testdir
    root@test:~/tartest# getfacl testdir
    # file: testdir
    # owner: root
    # group: root
    user::rwx
    group::r-x
    other::r-x

    root@test:~/tartest# tar cf testdir.tar testdir --acls
    root@test:~/tartest# rmdir testdir
    root@test:~/tartest# tar xf testdir.tar --acls
    root@test:~/tartest# ls -al | grep testdir$
    drwxr-xr-x+ 2 root root 2 Apr 1 13:59 testdir
    root@test:~/tartest#
    getfacl testdir
    # file: testdir
    # owner: root
    # group: root
    user::rwx
    group::r-x
    other::r-x
    default:user::rwx
    default:group::r-x
    default:eek:ther::r-x


    Having this default acl now set makes the directory behave differently.

    It seems this is a bug in gnu tar and was fixed back in 2014 (and thus in gnu tar 1.28):
    http://git.savannah.gnu.org/cgit/tar.git/commit/?id=7fe7adcbb985e78aaf9f78051fa26167779be1f6

    Unfortunately Jessie (which PVE is based on) is still using tar 1.27.1 which has this bug.

    This basically means that a container created in PVE by extracting a template using tar to create the root file system which has posixacls enabled will erroneously gain all these default acls due to this bug in tar because PVE uses the "--acls" option when it extracts the template (and this could cause unexpected behaviour for software in the container, particularly if there's any software which is using umask and not expecting to have default acls - and because freshly created directories inherit acls from the parent any directories created as part of package installs in the container are going to gain these default acls too). The only work around at the moment is to use setfacl (as noted above) to clear the default acls after the template has been extracted.
     
    #5 Skaffen, Apr 1, 2016
    Last edited: Apr 1, 2016
  6. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    15,347
    Likes Received:
    171
    Please can you report a bug at bugzilla.proxmox.com?
     
  7. Skaffen

    Skaffen New Member
    Proxmox VE Subscriber

    Joined:
    Apr 1, 2016
    Messages:
    3
    Likes Received:
    0
    Hi Dietmar - I've logged it there (bug 928) linking back to this thread. To sort the issue I suspect you'd have do your own tar package on PVE with either 1.27.1 patched with just that bug fix or backporting 1.28 from Stretch!
     
  8. manu

    manu Proxmox Staff Member
    Staff Member

    Joined:
    Mar 3, 2015
    Messages:
    716
    Likes Received:
    41
    Hi
    Thanks for the bug report, we're mostly finished on that.

    As a work around, please note that it's possible to disable to disable the use of ACLs inside a container by editing the root disk property in the GUI and disabling ACLs, or setting acl=0 in the pct config file.
    The acl=0 setting works on all storage expect ZFS at the momment, and a patch has been sent today on pve-devel to have this on ZFS too.
     
  9. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
    I'm experiencing this same problem with LXC containers on ceph.

    root@proxmoxnj2:~# pveversion -v
    proxmox-ve: 4.1-26 (running kernel: 4.2.6-1-pve)
    pve-manager: 4.1-1 (running version: 4.1-1/2f9650d4)
    pve-kernel-4.2.6-1-pve: 4.2.6-26
    lvm2: 2.02.116-pve2
    corosync-pve: 2.3.5-2
    libqb0: 0.17.2-1
    pve-cluster: 4.0-29
    qemu-server: 4.0-41
    pve-firmware: 1.1-7
    libpve-common-perl: 4.0-41
    libpve-access-control: 4.0-10
    libpve-storage-perl: 4.0-38
    pve-libspice-server1: 0.12.5-2
    vncterm: 1.2-1
    pve-qemu-kvm: 2.4-17
    pve-container: 1.0-32
    pve-firewall: 2.0-14
    pve-ha-manager: 1.0-14
    ksm-control-daemon: 1.2-1
    glusterfs-client: 3.5.2-2+deb8u1
    lxc-pve: 1.1.5-5
    lxcfs: 0.13-pve1
    cgmanager: 0.39-pve1
    criu: 1.6.0-1
    zfsutils: 0.6.5-pve6~jessie

    I added acl=0 in the container's config file like this:
    rootfs: proxmoxnj:vm-133-disk-1,size=8G, acl=0

    But ACL is still enabled and I'm still having problems with several apps.
     
  10. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    2,118
    Likes Received:
    242
    Please upgrade your installation and see if the issue persists - your versions are not current at all.
     
  11. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
    I'm on the latest version and the problem persists.

    root@proxmoxnj2:~# pveversion -v
    proxmox-ve: 4.1-26 (running kernel: 4.2.6-1-pve)
    pve-manager: 4.1-1 (running version: 4.1-1/2f9650d4)
    pve-kernel-4.2.6-1-pve: 4.2.6-26
    lvm2: 2.02.116-pve2
    corosync-pve: 2.3.5-2
    libqb0: 0.17.2-1
    pve-cluster: 4.0-29
    qemu-server: 4.0-41
    pve-firmware: 1.1-7
    libpve-common-perl: 4.0-41
    libpve-access-control: 4.0-10
    libpve-storage-perl: 4.0-38
    pve-libspice-server1: 0.12.5-2
    vncterm: 1.2-1
    pve-qemu-kvm: 2.4-17
    pve-container: 1.0-32
    pve-firewall: 2.0-14
    pve-ha-manager: 1.0-14
    ksm-control-daemon: 1.2-1
    glusterfs-client: 3.5.2-2+deb8u1
    lxc-pve: 1.1.5-5
    lxcfs: 0.13-pve1
    cgmanager: 0.39-pve1
    criu: 1.6.0-1
    zfsutils: 0.6.5-pve6~jessie
     
  12. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    2,118
    Likes Received:
    242
    those are not the latest versions. please follow the instructions at https://pve.proxmox.com/wiki/Downlo...Proxmox_Virtual_Environment_4.x_to_latest_4.1 and https://pve.proxmox.com/wiki/Package_repositories
     
  13. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
    Thanks Fabian. But my repo's are set up correctly and I just did an apt-get update and an apt-get dist-upgrade less than an hour ago. Then I rebooted the hosts. I'm not sure what else I can do to upgrade these servers.
     
  14. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
    We haven't activated the enterprise licenses yet on these servers because they just came out of our lab. Could the fix for this problem only be in the commercial repo?
     
  15. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    2,118
    Likes Received:
    242
    Please post the content of your apt sources files. My guess is you only have the enterprise repository enabled, but no subscription key configured, so you actually won't get any updates (in this case, apt-get update tells you that it cannot access the pve-enterprise repository).
     
  16. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
  17. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    2,118
    Likes Received:
    242
    Either activate your subscription key on that server, or enable the pve-no-subscription repository to get updates.

    If you enable the pve-no-subscription repository, you will get newer package versions than those included in pve-enterprise, so when you switch to pve-enterprise afterwards you will have to wait a bit until the enterprise repository has caught up with no-subscription (this is usually not a problem, no-subscription is just the last testing ground before a package migrates to enterprise and thus usually slightly ahead).
     
  18. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
    Upgraded to the latest versions.

    root@proxmoxnj1:~# pveversion -v
    proxmox-ve: 4.1-41 (running kernel: 4.2.8-1-pve)
    pve-manager: 4.1-22 (running version: 4.1-22/aca130cf)
    pve-kernel-4.2.6-1-pve: 4.2.6-36
    pve-kernel-4.2.8-1-pve: 4.2.8-41
    lvm2: 2.02.116-pve2
    corosync-pve: 2.3.5-2
    libqb0: 1.0-1
    pve-cluster: 4.0-36
    qemu-server: 4.0-64
    pve-firmware: 1.1-7
    libpve-common-perl: 4.0-54
    libpve-access-control: 4.0-13
    libpve-storage-perl: 4.0-45
    pve-libspice-server1: 0.12.5-2
    vncterm: 1.2-1
    pve-qemu-kvm: 2.5-9
    pve-container: 1.0-52
    pve-firewall: 2.0-22
    pve-ha-manager: 1.0-25
    ksm-control-daemon: 1.2-1
    glusterfs-client: 3.5.2-2+deb8u1
    lxc-pve: 1.1.5-7
    lxcfs: 2.0.0-pve2
    cgmanager: 0.39-pve1
    criu: 1.6.0-1


    Same issue.
     
  19. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    2,118
    Likes Received:
    242
    Could you post the output of "pct config ID" or the content of "/etc/pve/lxc/ID" on the host (replace ID with your container ID) and "mount" from within the container?
     
  20. axion.joey

    axion.joey Member

    Joined:
    Dec 29, 2009
    Messages:
    68
    Likes Received:
    1
    Here you go.

    root@proxmoxnj2:~# cat /etc/pve/lxc/100.conf
    arch: amd64
    cpulimit: 1
    cpuunits: 1024
    hostname: mcintirere.hivepbx.com
    memory: 512
    net0: bridge=vmbr0,gw=10.3.0.1,hwaddr=36:35:61:63:30:39,ip=10.3.0.50/21,name=eth0,type=veth
    onboot: 0
    ostype: centos
    rootfs: proxmoxnj:vm-100-disk-1,size=8G
    swap: 1024

    [root@ct100 ~]# mount
    /dev/rbd0 on / type ext4 (rw,relatime,stripe=1024,data=ordered)
    none on /dev type tmpfs (rw,relatime,size=100k,mode=755)
    proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
    proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
    proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
    proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
    sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
    sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
    sysfs on /sys/devices/virtual/net type sysfs (rw,relatime)
    sysfs on /sys/devices/virtual/net type sysfs (rw,nosuid,nodev,noexec,relatime)
    fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
    cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,size=12k,mode=755)
    tmpfs on /sys/fs/cgroup/cgmanager type tmpfs (rw,mode=755)
    lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/blkio type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/cpu,cpuacct type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/cpuset type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/devices type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/freezer type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/hugetlb type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/memory type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/systemd type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/net_cls,net_prio type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    lxcfs on /sys/fs/cgroup/perf_event type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
    devpts on /dev/lxc/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
    devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
    devpts on /dev/lxc/tty1 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
    devpts on /dev/lxc/tty2 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
    none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
    [root@mcintirere ~]#
     

Share This Page