[SOLVED] File ownership is sometimes incorrect when creating LXC

[Alex1]

My problem is that some images exported form LXD causes a problem with file ownership. It works if I use a image from linuxcontainers.org and upload it directly to proxmox, but not if I start the image on my workstation, export it and then upload the image to proxmox. (some images works just fine in both situations)

To recreate:

1. Install lxd(fast to install)
2. lxc launch images:ubuntu/xenial/amd64 test
   
3. lxc publish test --alias my-image --force
   
4. lxc image export my-image .
   
5. tar -xvzf <hash>.tar.gz
   
6. cd rootfs/
   
7. tar -zcvf ../ubuntu-16.04_my-image.tar.gz .
   
8. Upload the image(ubuntu-16.04_my-image.tar.gz) to Proxmox and start a container.

Some images(default LXD images, like ubuntu:xenial) works perfectly with this export method. The linuxcontainers.org images are more minimalistic compared to the default LXD ones, so I'm not sure if something is missing.

Output form ls / -la:

drwxr-xr-x  21 root   root    21 Apr 29 20:12 .
drwxr-xr-x  21 root   root    21 Apr 29 20:12 ..
drwxr-xr-x   2 ubuntu ubuntu 131 Apr 29 04:05 bin
drwxr-xr-x   2 ubuntu ubuntu   2 Apr 12 20:14 boot
drwxr-xr-x   7 root   root   480 Apr 29 20:12 dev
drwxr-xr-x  65 ubuntu ubuntu 129 Apr 29 20:12 etc
drwxr-xr-x   3 ubuntu ubuntu   3 Apr 29 04:06 home
drwxr-xr-x  11 ubuntu ubuntu  12 Apr 29 04:05 lib
drwxr-xr-x   2 ubuntu ubuntu   3 Apr 29 04:05 lib64
drwxr-xr-x   2 ubuntu ubuntu   2 Apr 29 04:05 media
drwxr-xr-x   2 ubuntu ubuntu   2 Apr 29 04:05 mnt
drwxr-xr-x   2 ubuntu ubuntu   2 Apr 29 04:05 opt
dr-xr-xr-x 908 root   root     0 Apr 29 20:12 proc
drwx------   3 ubuntu ubuntu   5 Apr 29 20:12 root
drwxr-xr-x  10 root   root   360 Apr 29 20:12 run
drwxr-xr-x   2 ubuntu ubuntu 114 Apr 29 04:06 sbin
drwxr-xr-x   2 ubuntu ubuntu   2 Apr 29 04:05 srv
dr-xr-xr-x  13 root   root     0 Apr 29 18:59 sys
drwxrwxrwt   7 root   root     7 Apr 29 20:37 tmp
drwxr-xr-x  10 ubuntu ubuntu  10 Apr 29 04:05 usr
drwxr-xr-x  11 ubuntu ubuntu  13 Apr 29 04:05 var

What can be the reason behind this file ownership problem?

 

[dietmar]

Do you use the latest version which includes a fixed tar (tar_1.27.1+pve.3)? I am not really sure if that is related.

 

[Alex1]

Yes, (Version: 1.27.1+pve.3), fully updated from pve-no-subscription.

I really don't get why it only happens with some containers, and only if it has been exported. It runs, but ownership is messed up.

After looking at the roadmap, might this be related to unprivileged containers? As far as I know both images are exported from unprivileged containers, but that still don't explain why the problem only exists with some images.

 

[dietmar]

but that still don't explain why the problem only exists with some images.

a bug in lxd export code?

 

[wbumiller]

Did you perform the above steps as root or as a user? Because I don't see you using the 'p' flag to preserve permissions/userids.
Not sure how their export works, but it looks like they're leaving the unprivileged user ids in the archive.
We expect the templates to be made from the perspective of the container with respect to user IDs.

 

[wbumiller]

Just tested, 'tar' reports the correct user-IDs on the exported image, so you probably tried to extract and create the archive as a user.
Either do it as root, or if that's not an option you can use lxc-usernsexec (you need the lxc1 package):

$ lxc image export myimage
$ uids="$(grep -F "$USER" /etc/subuid | cut -d: -f2-)"
$ gids="$(grep -F "$USER" /etc/subgid | cut -d: -f2-)"
$ lxc-usernsexec -m "b:0:$(id -u):1" -m "u:1:$uids" -m "g:1:$gids" -- /bin/bash
## Now you'll see yourself as root
root@host $ tar xpf <hash>.tar.gz rootfs/
## ignore errors about rootfs/dev/*, a user cannot create device nodes
root@host $ tar czpf template.tar.gz -C rootfs .

Alternatively you can go the tricky route as user without using user namespaces and modify your procedure as follows (you need to use bsdtar, not gnu tar, and the '@' in front of <hash>.tar.gz for the 3rd command is important):

$ lxc image export myimage
$ bsdtar xf <hash>.tar.gz rootfs
### extract permissions:
$ bsdtar cf perms.mtree --format=mtree --options='all' --include rootfs @<hash>.tar.gz
### fixup permission paths
$ sed -i -e 's@^./rootfs@.@' perms.mtree
### archive the data via the mtree file
$ bsdtar czf my-pve-template.tar.gz -C rootfs @../perms.mtree

Note that the latter version comes without promises

(I was actually going for a single bsdtar command to avoid unpacking at all, but none of the name modification options apply to files from @-included archives, so that's not an option unfortunately. Not with standard tools anyway.)

bsdtar and gnu tar aren't really compatible when it comes to extended attributes and such :-\

 

[Alex1]

Oh god, thank you so much for figuring it out wbumiller! I was going crazy trying to figure out what was going on. Running as root does of course make sense.