LXC and cset/cpuset quirk - all apparmor processes unconfined

[bladenj]

Hi

I'm using proxmox to host multiple LXC's and VM's, in order to get good gaming performance on my windows VM I am using cset and taskset to pin the windows cores to the last 8c/16t of my cpu. I have run into an issue with cset and lxc containers where if I define a slice for the windows VM LXC containers get access to all of the cores on the system slice and then apparmor runs everything in unconfined mode.

I run this command at boot as I don't mind losing 8c/16t as long as they get pinned:
cset set -c 0-31 -s machine.slice && cset shield --kthread on --cpu 8-15,24-31 && cset proc --move --fromset=root --toset=system --threads --kthread --force

This is the cset command that is run at launch with hookscripts:
cset proc --move --pid "$CPU_TASK" --toset=user --force

I have attached a few screenshots of what the lxc slices should look like compared to what they look like if I define my own cgroups.

Any help would be appreciated and thanks in advance.

 

[bladenj]

Also a note to add if I don't add this as a hookscript to every container I can only start the container once and otherwise would get an error

#!/bin/bash
VMID="$1"
VM_ACTION="$2"
if [[ "$VM_ACTION" == "post-stop" ]]; then
    sleep 1s
    cset set -d "$VMID" -r --force
    sleep 1s
fi

cg_hybrid_get_controllers: 657 Found hierarchy not under /sys/fs/cgroup: "/cpusets rw,relatime master:130 - cgroup none rw,cpuset"
cgfsng_monitor_create: 1365 Numerical result out of range - Failed to create monitor cgroup
__lxc_start: 1911 Failed to create monitor cgroup
TASK ERROR: startup for container '300' failed

 

[pantherts]

Hey there,

I just wanted to bump this thread as I have the exact same setup as you regarding your core designation (I have a 5950x with the second CCX set aside for my Windows VM on boot). I'm also using cpuset and taskset in my hookscript to push the tasks onto the cgroup I created for the VM. My LXC containers also have trouble starting due to the same errors you're receiving having to do with cgroups.

I'm using the kernel command line "systemd.unified_cgroup_hierarchy=1" in my default grub for the host. I've also tried adding: "lxc.init.cmd: /lib/systemd/systemd systemd.unified_cgroup_hierarchy=1" to my LXC container config file to no avail. It seems I have the same issue where if I reboot the host or shutdown everything and wipe all cgroups then the LXC containers will start back up.

Any clarification on all of this from anyone who may be experiencing similar issues would be super helpful. Thanks so much!