Map with write permissions a Proxmox folder inside a container that runs docker

[bluepr0]

Hi!

I'm in the process of optimizing my server and want to move some stuff from VMs to Docker. I'm running Proxmox 7.2-11.

- I've created an LXC container with Debian 11 and installed it inside docker with portainer
- I'm running a VM with TrueNAS where I've shared an NFS volume to Proxmox
- I've mounted this share at /mnt/Photos, so I can access it from any LXC

I've been reading A TON of posts about mapping and this is how my config looks like right now. I've followed this guide, Method 2

Host-side
LXC 101 Config

arch: amd64
cores: 2
features: nesting=1,keyctl=1
hostname: docker
memory: 2048
mp0: /mnt/pve/Photos,mp=/mnt/ncdata
net0: name=eth0,bridge=vmbr0,hwaddr=8E:4B:CF:E8:E1:46,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-101-disk-0,size=24G
swap: 512
unprivileged: 1
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.idmap: u 0 100000 1000
lxc.idmap: g 0 100000 1000
lxc.idmap: u 1000 1000 1
lxc.idmap: g 1000 1000 1
lxc.idmap: u 1001 101000 64535
lxc.idmap: g 1001 101000 64535

Permissions of the share

root@pve:~# ls -l /mnt/pve
total 17
drwxrwxrwx 7 1000 1000    9 Nov 14 11:21 Photos

Contents of /etc/subuid

root@pve:~# cat /etc/subuid
root:100000:65536
root:1000:1
root@pve:~#

Contents of /etc/subgid

root@pve:~# cat /etc/subgid
root:100000:65536
root:1000:1
root@pve:~#

Guest-side
I've added an user with ID 1000 with useradd -u 1000 -m -s /usr/bin/bash host-data

Then I check the permissions of the share

root@docker:~# ls -l /mnt/   
total 9
drwxrwxrwx 7 host-data host-data 9 Nov 14 10:21 ncdata

And I can also read and write files without any problems

root@docker:/mnt/ncdata# ls -l
total 2
drwxrwxrwx 2 host-data host-data 2 Nov 12 15:29 dump
drwxrwxrwx 2 host-data host-data 2 Nov 13 08:34 images
drwxrwxrwx 2 host-data host-data 2 Nov 12 15:29 private
drwxrwxrwx 4 host-data host-data 4 Nov 12 15:29 template
root@docker:/mnt/ncdata# mkdir test
root@docker:/mnt/ncdata# ls -l
total 3
drwxrwxrwx 2 host-data host-data 2 Nov 12 15:29 dump
drwxrwxrwx 2 host-data host-data 2 Nov 13 08:34 images
drwxrwxrwx 2 host-data host-data 2 Nov 12 15:29 private
drwxrwxrwx 4 host-data host-data 4 Nov 12 15:29 template
drwxrwxrwx 2 nobody    host-data 2 Nov 14 13:22 test
root@docker:/mnt/ncdata#

PROBLEM
While looks like I can read and write files, when I install portainer and then nextcloud docker, at the moment of installation it says it can't write to /mnt/ncdata

Here's my docker-compose

version: '3'

services:
  db:
    image: mariadb:10.5
    command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
    restart: always
    volumes:
      - /root/docker/nextcloud/db:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=nextcloud
      - MARIADB_AUTO_UPGRADE=1
      - MARIADB_DISABLE_UPGRADE_BACKUP=1
    env_file:
      - stack.env

  redis:
    image: redis:alpine
    restart: always

  app:
    image: nextcloud:apache
    restart: always
    ports:
      - 8080:80
    volumes:
      - /root/docker/nextcloud/nextcloud:/var/www/html
    environment:
      - MYSQL_HOST=db
      - REDIS_HOST=redis
      - NEXTCLOUD_UPLOAD_LIMIT=300G
      - NEXTCLOUD_MEMORY_LIMIT=2048M
      - NEXTCLOUD_DATA_DIR=/mnt/ncdata
    env_file:
      - stack.env
    depends_on:
      - db
      - redis

  cron:
    image: nextcloud:apache
    restart: always
    volumes:
      - /root/docker/nextcloud/nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db
      - redis

If I install nextcloud without NEXTCLOUD_DATA_DIR variable so the data is also installed in the same place it runs just fine, but I want to use for my data the /mnt/Photos share as it's running on my TrueNAS with protection and backups.

I've been trying for several days and I'm a bit lost on why I can read and write from the LXC console but not when running from docker. I've checked and the docker containers seems to be running as root (at least what it says on htop)

 

[bluepr0]

I made a test and initiated an uptime-kuma docker and mapping its volume to /mnt/ncdata/uptime and it worked fine. So looks like docker can write to the folder

Further investigation and I found this thread https://stackoverflow.com/a/68855899/1546357 it might be related that the Nextcloud container doesn't use root to initiate it?

 

[bluepr0]

I've added the user www-data to host-data group which is 1000 but doesn't seem to work. I'm not sure if this should work but I've made so many changes that I messed up something or if I'm still doing something wrong or what