UID/GID mapping breaking in-container UID/GIDS?

[flypenguin]

hi all, i am migrating to jellyfin as a kodi replacement, and for this i intended to use the following setup:

1. a jellyfin LXC backend (naturally), installed via community scripts
2. a "NAS" container to manage my files via web / SMB / ...
3. ... both accessing the same files on the host via bind mount.
4. the files on the host are mounted with nobody:nobody permissions, because there is no clear "owner" on the host system.

that might also be a template for further applications like ente, immich, ... . the diagram below shows the idea.

now the first milestone was to get jellyfin and the media access up and running, including the UID/GID mapping, so the required mapping is:

"HOST user nobody (65534:65534)" to "CONTAINER user jellyfin (107:110)"

Result: The mapping of the bind mount works, but now the UID of in-container files/directories is broken – jellyfin-owned files get mapped to "nobody"
(see "THE ERROR" below)

could anyone tell me why, or even better – how to "fix" this? because jellyfin will of course not run in this situation, and this feels like "that's not the way this is intended to be ...".

what i have unsuccessfully tried so far:

• pretty much any combination within the xxx.conf file, either the system won't start, or the mapping will not work. (of course maybe i missed the EXACT right combination, sure, i just tried a lot of them ... )
• adding "root:110:1" and/or "root:107:1" to /etc/set(u|g)id, but that did not help either.
• reading tons of blog posts about the subject

thanks in advance!
axel.


THE ERROR

# note the "nobody adm" uid/gids below
# WRONG UID
root@jelly:~# ll /var/lib/jellyfin -lad
drwxr-x--- 9 nobody adm 4096 Oct 17 21:43 /var/lib/jellyfin/

# after I REMOVING THE ID MAPPING from /etc/pve/lxc/...conf:
# CORRECT UID
root@jelly:~# ll /var/lib/jellyfin -lad
drwxr-x--- 9 jellyfin adm 4096 Oct 17 21:43 /var/lib/jellyfin/

THE PART WORKING

# WITH the uid/gid mappings, the bind-mounted directory shows up correctly:

root@jelly:~# ll /srv/media/
drwxr-xr-x 2 jellyfin jellyfin    6 Jan 16 22:01 Movies/
drwxr-xr-x 2 jellyfin jellyfin    6 Jan 16 22:01 TvShows/

root@jelly:~# ll -n /srv/media/
drwxr-xr-x 2 107 110    6 Jan 16 22:01 Movies/
drwxr-xr-x 2 107 110    6 Jan 16 22:01 TvShows/


# after REMOVING THE UID/GID MAPPINGS, the bind mount's UID/GID are, expectedly, incorrect again

root@jelly:~# ll /srv/media/
drwxr-xr-x 2 nobody nogroup    6 Jan 16 22:01 Movies/
drwxr-xr-x 2 nobody nogroup    6 Jan 16 22:01 TvShows/

root@jelly:~# ll -n /srv/media/
drwxr-xr-x 2 65534 65534    6 Jan 16 22:01 Movies/
drwxr-xr-x 2 65534 65534    6 Jan 16 22:01 TvShows/

THE CONF / SETUID / SETGID FILES

# xxx.conf
# non-important stuff removed
arch: amd64
# [...]
mp0: /srv/media,mp=/srv/media
lxc.idmap: u 0 100000 107
lxc.idmap: u 107 65534 1
lxc.idmap: u 108 100108 65428
lxc.idmap: g 0 100000 110
lxc.idmap: g 110 65534 1
lxc.idmap: g 111 100111 65425

# /etc/subuid
root:100000:65536
root:65534:1

# /etc/subgid
root:100000:65536
root:65534:1

THE SETUP

┌─────────────────────────────────────┐
│                                     │
│   ┌──────────┐    ┌────────────┐    │
│   │          │    │            │    │
│   │ Jellyfin │    │ "NAS" LXC  │    │
│   │          │    │            │    │
│   └┬─────────┘    └───────────┬┘    │
│    │uid/gid         uid/gid   │     │
│    │107:110         ???:???   │     │
│    │                          │     │
│    │                          │     │
│    │       nobody:nobody      │     │
│    │       65534:65534        │     │
│    └─────► /srv/media ◄───────┘     │
│                                     │
│    (Host-mounted media folder)      │
│                                     │
│Host                                 │
└─────────────────────────────────────┘

 

[flypenguin]

so yeah, i guess the solution is "this does not work this way", at least according to loooong chats with chatgpt and way too many tries and restarts.

it seems the second you bind the host's UID1 to a container's UID2, you apparently bind the container's _original_ UID2 to the hosts's UID1. (wha ... ? yeah, read a couple of times .

example:

• binding 1000 (host) -> 2000 (container) will apparently (correctly) let you show files owned by 1000 in the container by uid 2000.
• BUT AT THE SAME TIME apparently the container's original UID 2000 will now be 1000, so your uid 1000 user's home directory is no longer accessible for itself. which is ... i don't know what this is, i just know it is the opposite of "useful".

and should this be in fact the case I would find that utterly idiotic (i really cannot imagine any realistic scenario where this does not shoot you in the foot), BUT this is the only way I got this "working" at all. and chatgpt agrees. the documentation on the whole phenomenon is, to be frank, pure shit, or should in fact some useful documentation exist then the "findability" of it is.

BUT: I found no explicit sources for this, so I can be completely wrong. but I wanted to write something for other "me"s that are hitting that page from google, in the search, or whatever. because also frankly the available documentation for the whole topic is a pile of ...., and most of them have a common error with the subuid/gid files it seems.

if i indeed got it wrong and anyone can figure out a working solution for this, or an explanation why this in fact makes sense, i would highly appreciate a comment here or a message – because i truly do not see any actual practical use for such a seemingly strange mechanism except for some super niche use cases.

 

[Gavino]

Brother - I feel your pain. I am having the same sorts of issues as you but I have Plex Media server using the community scripts instead of Jellyfin. My NAS container is Cockpit with the 45drives plugins.

I have my bind-mount being owned by 1090:1090 UID/GID. The community scripts set up Plex as 999/990 UID/GID. I follow all the docs and was celebrating when I saw that after I did the mapping, the bind mount and all files within were showing up in the Plex LXC as 999:990 (plex: plex), and I could read and write to them fine within the LXC. But then plex doesn't run because everything in /var/lib/plexmediaserver is now owned by 65534:65534 (nobody:nogroup) instead of what it had previously before mapping, which is 999:990 (plex: plex).

What the hell? Nowhere in docs state that mapping outside UID/GID (outside of LXC) to inside UID/GID (inside of LXC) screws up UID/GID inside the LXC!

So I can either get Plex working (default install) but no container access to the media, or access to the media, but no working Plex. This is madness. I have been pulling my hair out about this for the last 24 hours.

I set up a test user within the LXC giving it 1090:1090 but /var/lib/plexmediaserver is still owned by 65534:65534. I noticed that when I set up this test user, that it added a "testuser:100000:65536" to the /etc/subuid and /etc/subgid within the LXC container.

I have no idea how to fix this. If anyone can shed light on this - I would be so appreciative. This is going to drive me crazy for days! I might have to just give up and run a privileged container, as unprivileged containers seems like a hot mess / steaming dumpster fire.

 

[Gavino]

Success!!

What fixed it for me was changing the GID/UID of the "plex" user within the container, and changing ownership of files. I'll map this all out for prosperity.

Background:
On the host I have all my media files in an share where the user is 1090:1090 when I copy files onto the host using SMB. I want to Integrate Plex with that and not have permissions screw ups, but I also want to run Plex in an unprivileged container. When I install Plex LXC using the community scripts, there is a "plex" user and "plex" group within the LXC, with UID/GID of 999 (UID) and 990 (GID).

Procedure:
1) Install Plex LXC as to your preferences using the script. For me it's LXC ID of 100.

2) stop the container "pct stop 100"

3) Edit the container config. "vi /etc/pve/lxc/100.conf"
For my mappings of 1090:1090 I append this following:

lxc.idmap: u 0 100000 1090
lxc.idmap: g 0 100000 1090
lxc.idmap: u 1090 1090 1
lxc.idmap: g 1090 1090 1
lxc.idmap: u 1091 101091 64445
lxc.idmap: g 1091 101091 64445

I also add the UID/GUID to the subuid and subgid files...

root@gtnas01:/var/lib# cat /etc/subuid
root:1090:1
root:100000:65536

root@gtnas01:/var/lib# cat /etc/subgid
root:1090:1
root:100000:65536

I read a blog post somewhere that says to add them first before the "root:100000:65536" so I did that. Not sure if it matters but just in case I have put them first.

4) start her up again with "pct start 100"
I can check that the mappings are golden with:

root@gtnas01:/var/lib# pct exec 100 -- cat /proc/self/uid_map
         0     100000       1090
      1090       1090          1
      1091     101091      64445
root@gtnas01:/var/lib#
root@gtnas01:/var/lib# pct exec 100 -- cat /proc/self/gid_map
         0     100000       1090
      1090       1090          1
      1091     101091      64445

5) now enter the container "pct enter 100".
I can see Plex is running with "systemctl status plexmediaserver" and it's using the old uid/gid

root@plex:/var/lib# cat /etc/passwd | grep plex
plex:x:999:990::/var/lib/plexmediaserver:/usr/sbin/nologin

Stop the server with "systemctl stop plexmediaserver"

6) update plex UID / GID

usermod -u 1090 plex    #1090 is new UID
groupmod -g 1090 plex   #1090 is new GID
root@plex:/var/lib# cat /etc/passwd | grep plex
plex:x:1090:1090::/var/lib/plexmediaserver:/usr/sbin/nologin

# Substitute below IDs for the actual previous values
find / -user 999 -exec chown -h plex {} \;
find / -group 990 -exec chgrp -h plex {} \;

7) Now I bind mount and set permissions

pct set 100 -mp0 /mnt/pve/multimedia,mp=/mnt/multimedia
chown -R 1090:1090 /mnt/pve/multimedia/

Final Checks
After restarting the container I can see that from inside the container, my /mnt/multimedia has all the right permissions and plex media server is up and running fine. so to me it seems broken when the IDs inside and outside the container are low IDs that don't match. By that I mean don't match stuff like 1001 outside the container to say 999 inside the container. Make the low IDs the same on both. Even if it means you have to change UID/GID for the main process inside the container. That is what worked for me.