Tutorial: Run LLMs using AMD GPU and ROCm in unprivileged LXC container

Dirty snippet with arch container:
Code:
nano /etc/pacman.d/mirrorlist   /2-3 server (https!)
pacman-key --init && pacman-key --populate archlinux && pacman-key --refresh-keys
pacman -Syu openssh sudo && systemctl restart sshd && systemctl enable sshd
visudo
################
## Uncomment to allow members of group wheel to execute any command
 %wheel ALL=(ALL:ALL) ALL      //(x + :wq)

useradd -m -G wheel,users,render,video -s /bin/bash archc01
passwd archc01

*ab hier via SSH*

sudo pacman -S --needed base-devel git rocminfo
git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si && yay --version
yay python312

auf host:
ls -l /dev/dri/by-path/ /dev/kfd /dev/dri && lspci -d ::03xx

/dev/kfd -> mapping to render GID of LXC
/dev/dri/renderD128 -> mapping to video GID of LXC
/dev/dri/renderD129 -> mapping to video GID of LXC

in container:
cat /etc/group | grep -w 'render\|\video'

shutdown container, restart after setting passthrough

git clone https://github.com/vladmandic/sdnext
cd sdnext
python3.12 -m venv venv
./webui.sh --listen --debug --auth "CreativeOne:p4sSw0rD" --use-rocm
or
./webui.sh --listen --debug --auth "CreativeOne:p4sSw0rD" --use-zluda

sit back and wait

browse to http://IP:7860

Btw. no need to set those variables, it is autodetection and just works.
Code:
HSA_OVERRIDE_GFX_VERSION=9.0.0
HSA_ENABLE_SDMA=0
Bildschirmfoto zu 2025-02-15 11-34-04.png
Bildschirmfoto zu 2025-02-15 11-35-10.png
Bildschirmfoto zu 2025-02-15 11-36-00.png
 
Last edited: