Migrating plain unprivileged LXC container to proxmox's ones

[sammyjenkins]

I'm trying to migrate unprivileged lxc container (created not on proxmox) to proxmox.

I saw couple of posts about how to do that kind of thing:
https://serverfault.com/questions/7...ular-lxc-container-to-a-proxmox-lxc-container
https://forum.proxmox.com/threads/import-non-pve-created-lxc-container-into-pve-5-2.49049/

As I understood I can just tar the container, move it to proxmox host and create a new one using `pct create`.

So that's what I did:
1. Created tar (tar -czvf ./ct.tar.gz /var/lib/lxd/containers/container1/rootfs)
2. Moved the tar to the proxmox host
3. Created a disk (I'm using ZFS storage):

    zfs create storage/subvol-100-disk-0
    zfs set xattr=sa storage/subvol-100-disk-0
    zfs set acltype=posixacl storage/subvol-100-disk-0

4. Using pct created a container:
pct create 100 /ct.tar.gz -description container1 -hostname container1 -memory 1024 -nameserver 8.8.8.8 -storage storage -password changeme --rootfs storage:subvol-100-disk-0,quota=0

Container is created, no errors in logs.
I can start the container, but proxmox shows that container now is privileged, though if I list rootfs files, it shows unprivileged ids (100000 : ...).
Container also acts weird, I can't upgrade some packets, stop and start is slow. My guess is that it somehow connected to that privileged state.

So, my question is how can I say proxmox that container is already in an unprivileged mode? Or am I doing something completely wrong?
I also tried to explicitly say to proxmox, that container should be created in an unprivileged mode using flag --unprivileged 1 in pct command, but it throws a bunch of Permission denied errors and fails to create a container.

Thank you very much for the help in advance.

 

[oguz]

hi,

your steps look good.

I also tried to explicitly say to proxmox, that container should be created in an unprivileged mode using flag --unprivileged 1 in pct command, but it throws a bunch of Permission denied errors and fails to create a container.

it'd help if you posted the output, but my guess is your container has some device nodes (mknod: Permission denied ?), and in unprivileged mode you're not allowed to create those. you can try removing them from the tar archive and try again.

 

[sammyjenkins]

Hello, Oguz!

Thank you very much for the response!
Here's part of the log:

tar: lib: Cannot mkdir: Permission denied
tar: lib: Cannot mkdir: Permission denied
tar: lib/libhandle.so.1.0.3: Cannot open: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/modules: Cannot mkdir: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/terminfo: Cannot mkdir: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/terminfo/l: Cannot mkdir: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/terminfo/l/linux: Cannot open: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/terminfo/x: Cannot mkdir: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/terminfo/x/xterm-xfree86: Cannot open: No such file or directory
tar: lib: Cannot mkdir: Permission denied
tar: lib/terminfo/x/xterm-256color: Cannot open: No such file or directory
tar: lib: Cannot mkdir: Permission denied
...

tar: sys: Cannot mkdir: Permission denied
tar: mnt: Cannot mkdir: Permission denied
tar: tmp: Cannot mkdir: Permission denied
tar: tmp: Cannot mkdir: Permission denied
tar: tmp/.X11-unix: Cannot mkdir: No such file or directory
tar: tmp: Cannot mkdir: Permission denied
tar: tmp/.XIM-unix: Cannot mkdir: No such file or directory
tar: tmp: Cannot mkdir: Permission denied
tar: tmp/.Test-unix: Cannot mkdir: No such file or directory
tar: tmp: Cannot mkdir: Permission denied
tar: tmp/.font-unix: Cannot mkdir: No such file or directory
tar: tmp: Cannot mkdir: Permission denied
tar: tmp/.ICE-unix: Cannot mkdir: No such file or directory
tar: tmp: Cannot mkdir: Permission denied
tar: tmp/gitaly-ruby873599363: Cannot mkdir: No such file or directory
Total bytes read: 6692259840 (6.3GiB, 81MiB/s)
tar: Exiting with failure status due to previous errors

So, as far as, mknod: Permission denied is not there, looks like device nodes are not the problem.
I checked that files are in place, by extracting tar by hands. Also if I create containers without --unprivileged 1 flag, all the files are in place.

 

[fabian]

your tar probably contains the wrong ownership information.

pct expects a template/backup archive to contain the UID/GID view from inside the container (i.e., root = 0). you created the tar from the host, where container's root = 100000 (or whatever mapping your lxd container had ). basically, you'd need to create your tar from within the container, or at least from within a user namespace with the same mapping as the container (e.g., with "lxc-usernsexec").

 

[sammyjenkins]

I see, that makes sense!
Thank you very much for the advice, I'll try to do that in weekends and get back to you

 

[sammyjenkins]

Thank you, Fabian!
That worked.
Now, I have a privileged container.
I read that, by simply, backing up and restoring I can make it unprivileged. Though when I do it, I still get Permission denied errors:

# pct restore 103 /var/lib/vz/dump/vzdump-lxc-100.tar.lzo --storage storage --rootfs storage:subvol-100-disk-0,quota=0 -ignore-unpack-errors 1 -unprivileged
extracting archive '/var/lib/vz/dump/vzdump-lxc-100.tar.lzo'
tar: ./etc: Cannot mkdir: Permission denied
tar: ./etc/vzdump/pct.conf: Cannot open: No such file or directory
tar: ./var: Cannot mkdir: Permission denied
tar: ./var: Cannot mkdir: Permission denied
tar: ./var/cache: Cannot mkdir: No such file or directory
tar: ./var: Cannot mkdir: Permission denied
...

and at the end it says:
Backup archive does not contain any configuration

Am I doing it wrong?


Thank you very much for the help!

 

[fabian]

you should use "pct create" and not "pct restore" if your archive only contains the content. restore expects content + config, as created by vzdump.

can you try a regular creation:

"pct create XXX /var/lib/vz/dump/vzdump-lxc-100.tar.lzo --rootfs storage:8 -unprivileged"

and verify with "tar -tvf /var/lib/vz/dump/vzdump-lxc-100.tar.lzo" that ownership/permissions inside the tar archive look correct? most files should be owner by 0/0 (numeric).

can you give the tar command you used to create your latest archive? as inspiration, you can look at what vzdump does, maybe you are missing something important:
https://git.proxmox.com/?p=pve-cont...e4e6d553c905834eaf3854acbebbf347;hb=HEAD#l338
https://git.proxmox.com/?p=pve-stor...d3a5027a83b569b9259a;hb=refs/heads/master#l18

 

[sammyjenkins]

Awesome! Thank you very much Fabian!


pct create XXX /var/lib/vz/dump/vzdump-lxc-100.tar.lzo --rootfs storage:8 -unprivileged gave me mknod: Operation not permitted errors.
-ignore-unpack-errors 1 resolved the issue with mknod and the container was created as an unprivileged one.

The one thing I didn't understand is that, I created a backup (tar) using web interface of proxmox, so shouldn't pct restore work in that case?
When I go to Container -> Backup -> Show Configuration proxmox shows that backup file, that I tried to restore from, has configuration bound to it.

and verify with "tar -tvf /var/lib/vz/dump/vzdump-lxc-100.tar.lzo" that ownership/permissions inside the tar archive look correct? most files should be owner by 0/0 (numeric).

Permissions inside the tar looks good

 

[fabian]

Awesome! Thank you very much Fabian!


pct create XXX /var/lib/vz/dump/vzdump-lxc-100.tar.lzo --rootfs storage:8 -unprivileged gave me mknod: Operation not permitted errors.
-ignore-unpack-errors 1 resolved the issue with mknod and the container was created as an unprivileged one.

The one thing I didn't understand is that, I created a backup (tar) using web interface of proxmox, so shouldn't pct restore work in that case?
When I go to Container -> Backup -> Show Configuration proxmox shows that backup file, that I tried to restore from, has configuration bound to it.

yes, a backup archive created with vzdump should not print the message "Backup archive does not contain any configuration".. I misread your post - the error message appeared because it was unable to extract ANY files, and thus also not the contained configuration..

 

[sammyjenkins]

That's strange..
Anyway, thank you very much!