snaps in lxc on proxmox 7.1

[Pedulla]

Snapd installs in an lxc with fuse and squashfuses and fuse installed first (and a few conf file adds), but when you go to run a snap, you get:
"cannot open freezer cgroup (/sys/fs/cgroup/freezer): No such file or directory"

I suspect it has something to do with the v2 cgroups thing, but no idea how to fix/work around it.

Anyone else seeing this and know how to get snaps to run in an lxc on proxmox 7.1?

 

[oguz]

hi,

it works here using an unprivileged ubuntu container. try like the following (where 123 is the container ID):

$ pct set 123 -features nesting=1,fuse=1
$ pct reboot 123 || pct start 1022
$ pct enter 123
# apt install snapd
# snap install nextcloud
2021-11-29T12:19:36Z INFO Waiting for automatic snapd restart...
Warning: /snap/bin was not found in your $PATH. If you've not restarted your session since you
         installed snapd, try doing that. Please see https://forum.snapcraft.io/t/9469 for more
         details.

nextcloud 22.2.3snap1 from Nextcloud✓ installed

and it started correctly. here's my container config:

arch: amd64
cores: 1
description: ubuntu-20.10-standard_20.10-1_amd64.tar.gz
features: nesting=1,fuse=1
hostname: CT123
memory: 512
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=XX:XX:XX:XX:XX:XX,ip=dhcp,type=veth
ostype: ubuntu
rootfs: local-btrfs:1022/vm-123-disk-0.raw,size=4G
swap: 512
unprivileged: 1

 

[Pedulla]

Thanks, I'll try a Ubuntu container. I was using Debian.
In the Debian container, I had to install squashfuse and fuse before snapd would install.
I'll report back.
Thanks

 

[Pedulla]

Okay, switched over to use the Ubuntu 20.04 template and snapd install w/o issue. Matched what you did @oguz and nextcloud and hello-world install just fine. Ubuntu apparently has all the squashfs stuff built in where Debian didn't. Okay.

My real need here is to run the snap of ffmpeg to take advantage of my nvidia gpu passthrough.
When I go to install "snap install ffmpeg" I get

snap install ffmpeg
error: cannot perform the following tasks:
- Setup snap "ffmpeg" (1286) security profiles (cannot setup udev for snap "ffmpeg": cannot reload udev rules: exit status 1
udev output:
Failed to send reload request: No such file or directory
)
- Setup snap "ffmpeg" (1286) security profiles (cannot reload udev rules: exit status 1
udev output:
Failed to send reload request: No such file or directory
)
- Setup snap "ffmpeg" (1286) security profiles for auto-connections (cannot reload udev rules: exit status 1
udev output:
Failed to send reload request: No such file or directory
)

my config is as follows:

arch: amd64
cores: 4
features: fuse=1,nesting=1
hostname: TestGPU1
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=1E:23:B3:42:45:70,ip=dhcp,tag=33,type=veth
ostype: ubuntu
rootfs: ISOs:subvol-114-disk-0,size=8G
swap: 512
unprivileged: 1
lxc.cgroup.devices.allow: c 195:* rwm
lxc.cgroup.devices.allow: c 235:* rwm
lxc.cgroup.devices.allow: c 238:* rwm
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/nvidia-caps dev/nvidia-caps none bind,optional,create=dir
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,cr

Also there's this in syslog during the install:

Nov 29 20:32:54 pve kernel: [89176.030964] audit: type=1400 audit(1638246774.291:367): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" label="lxc-114_</var/lib/lxc>//&:lxc-114_<-var-lib-lxc>:unconfined" name="snap.ffmpeg.ffmpeg" pid=1328088 comm="apparmor_parser"

Interestingly enough, I did not get this error on the Debian template.
So, what needs to be done to handle the udev errors?

 

[chriskirsche]

@Pedulla Were you able to fix this? I'm running into the same issue after upgrading from 6.4 to 7.1 last weekend.
Some of my containers with snap (like AdGuard) now have problems too. They will not start and run into file permission problems.

@oguz Might this be related to the Proxmox version?

To be more specific:

root@AdGuard-Home:~# snap install adguard-home
error: cannot perform the following tasks:
- Setup snap "snapd" (14549) security profiles (cannot reload udev rules: exit status 1
udev output:
Failed to send reload request: No such file or directory
)

 

[chriskirsche]

@Pedulla Were you able to fix this? I'm running into the same issue after upgrading from 6.4 to 7.1 last weekend.
Some of my containers with snap (like AdGuard) now have problems too. They will not start and run into file permission problems.

@oguz Might this be related to the Proxmox version?

To be more specific:

root@AdGuard-Home:~# snap install adguard-home
error: cannot perform the following tasks:
- Setup snap "snapd" (14549) security profiles (cannot reload udev rules: exit status 1
udev output:
Failed to send reload request: No such file or directory
)

I fixed the issue for me:

Added this line in the .conf file of the container

lxc.mount.auto: cgroup:rw
lxc.mount.auto: sys:rw

@Pedulla maybe you can try this. Would like to hear if this solves your problem too.

However I'm not sure about any implications this has. With the previous version of Proxmox I didn't had to have this in the .conf before.

Can someone in here explain this?

EDIT: If you want to find out if you are affected by this, check

service udev status in the container.

In my case the udev Kernel Device Manager failed to start with:
ConditionPathIsReadWrite=/sys was not met

 

[Pedulla]

@chriskirsche lxc.mount.auto: sys:rw was the ticket.
Had to remove and reinstall the snap, then joy!

I had the cgroup:rw already in the conf file as it is required for the nvidea passthrough. (just clipped it from my cut and paste :/)

Thank you SO much! Now I don't have to compile ffmpeg for nvec.

If anyone else knows, are there any security implications here?
(Seems like there might be)

 

[Pedulla]

Follow up:
After review https://www.man7.org/linux/man-pages/man5/lxc.container.conf.5.html
It seems you only need sys:rw for the install of the snap.
After it's installed you change to sys:ro (or mixed). Seems safer...

 

[chriskirsche]

@chriskirsche lxc.mount.auto: sys:rw was the ticket.
Had to remove and reinstall the snap, then joy!

I had the cgroup:rw already in the conf file as it is required for the nvidea passthrough. (just clipped it from my cut and paste :/)

Thank you SO much! Now I don't have to compile ffmpeg for nvec.

If anyone else knows, are there any security implications here?
(Seems like there might be)

Glad to hear it worked!

Maybe you should set the topic to closed ;-)

 

[chriskirsche]

Follow up:
After review https://www.man7.org/linux/man-pages/man5/lxc.container.conf.5.html
It seems you only need sys:rw for the install of the snap.
After it's installed you change to sys:ro (or mixed). Seems safer...

Thats a good hint. Thanks.

 

[Pedulla]

Maybe you should set the topic to closed ;-)

I'm holding out for some guidance on the security implications from anyone who is "in the know"...