Recommended way to use a HBA/SAS Expander in LXC

domi2

Member
Nov 8, 2022
7
1
8
I have been trying the past week to move most of my VMs over to LXC since they all run Linux anyway.
The only thing I'm struggling with is finding a easy solution to pass through my SAS expander or the block devices directly to the container so I can do all the mounting in there.

Obviously with VMs there is a simple PCI passthrough solution in Proxmox but it seems for LXC there is nothing similar.

The current solution I got working is this:

  • Let Proxmox handle the HBA
  • Create a privileged LXC container
  • Add to LXC config (where 65 is the major ID of my HBA block devices):
    Code:
    lxc.apparmor.profile: unconfined
    lxc.cgroup.devices.allow: a
    lxc.cap.drop:
    lxc.cgroup2.devices.allow: b 65:* rwm
    lxc.hook.autodev: /opt/hooks/lxc-storage-hook.sh
  • Create hook script:
    Code:
    #!/bin/sh
    
    mknod -m 0666 ${LXC_ROOTFS_MOUNT}/dev/drive01 b 65 0
    ...
    ...
  • Add every drive to the hook script

Now this works perfectly fine, but it requires you to add every device to the hook script, which is a pain when there are 24+ drives connected to the expander.

I also tried mounting /dev/disk/by-id in the container but that obviously won't work because they are symlinks to /dev/sdX, which can change on every boot so I can't link them to the container.

Trying to passthrough the PCI nodes didn't quite work either since the mpt3sas kernel module can't be loaded in the container.

Is this the best solution for LXC or is there a better way to achieve this?
Ideally the Proxmox host wouldn't even "see" the drives as to not clutter the web disk view. Currently they get automatically created.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!