Proxmox 8.4 Lxc Ollama Amd APU support

K

Keugon

New Member
Jul 2, 2025
7
0
1
Hardware:
Amd Ryzen 4600g
Proxmox verison:
pve-manager/8.4.13/5b08ebc2823dd9cb (running kernel: 6.8.12-15-pve)

Ich versuche bei einem Ollama LXC container mit dem Community script installiert:
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/ollama.sh)"
Das problem ist das es immer via CPU betrieben wird spricht.
ollama ps
gibt 100% CPU bei Prozessor zurück was man in der leistung auch gut merkt.

PS: Ich habe alle Tutorials durchgesucht was ich gefunden habe aber kaum eines beschreibt das ganze mit einer AMD APU ich hoffe ich übersehe ihr nur was kleines bin auch kein profi in sachen Linux zumindest das vorgehen hier gibt mir den eindruck :/

Übersehe ich hier etwas das ich an pve node noch machen muss damit es im Container geht?




LXC config:
Code: 
arch: amd64
cores: 10
features: nesting=1,fuse=1
hostname: ollama
memory: 8192
net0: name=eth0,bridge=vmbr0,gw=192.168.178.1,hwaddr=BC:24:11:75:F5:98,ip=192.168.178.216/24,type=veth
onboot: 1
ostype: ubuntu
parent: amdgppuinstallnodkms
rootfs: local-zfs:subvol-116-disk-0,size=65G
swap: 512
tags: ai;community-script
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.cgroup2.devices.allow: c 188:* rwm
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/serial/by-id  dev/serial/by-id  none bind,optional,create=dir
lxc.mount.entry: /dev/ttyUSB0       dev/ttyUSB0       none bind,optional,create=file
lxc.mount.entry: /dev/ttyUSB1       dev/ttyUSB1       none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM0       dev/ttyACM0       none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM1       dev/ttyACM1       none bind,optional,create=file
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

Wenn ich RocM lt. Rocm install vorgehe kommt beim prüfen von rocminfo:

Code: 
ROCk module is loaded
Unable to open /dev/kfd read-write: No such file or directory
root is member of video group

was ich als kernproblem vermute.

Vainfo:

Code: 
root@ollama:~# vainfo
Trying display: wayland
Trying display: x11
error: can't connect to X server!
Trying display: drm
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_20
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: Mesa Gallium driver 25.0.7-0ubuntu0.24.04.2 for AMD Radeon Graphics (radeonsi, renoir, ACO, DRM 3.57, 6.8.12-15-pve)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

Devices:
Code: 
root@ollama:~# ls -l /dev/dri/
total 0
drwxr-xr-x 2 root root        80 Sep 18 18:36 by-path
crw-rw---- 1 root video 226,   1 Sep 18 18:36 card1
crw-rw---- 1 root _ssh  226, 128 Sep 18 18:36 renderD128
 
Keugon said:
was ich als kernproblem vermute.
Click to expand...
Jep, /dev/kfd muss auch im LXC sein.

Wegen Berechtigungen:
S

Thread 'Tutorial: Run LLMs using AMD GPU and ROCm in unprivileged LXC container'

Create a Ubuntu 24.04 LXC container.
I used the excellent tteck script but you can also do using any other method you are comfortable with.
Give it plenty of specs regarding storage, RAM and CPU (according to Ollama's recommendations)
I chose 32GB and all available cores. Give it plenty of storage to accomodate your models (can be expanded using the GUI)
I chose 60 GB to start. ROCm takes up ~30 GB.

In the LXC container:
install ROCm according to official instructions but do not use DKMS as the drivers are already installed on Proxmox by default.
This is the purpose of...
  • Like
 
mr44er said:
Jep, /dev/kfd muss auch im LXC sein.

Wegen Berechtigungen:
S

Thread 'Tutorial: Run LLMs using AMD GPU and ROCm in unprivileged LXC container'

Create a Ubuntu 24.04 LXC container.
I used the excellent tteck script but you can also do using any other method you are comfortable with.
Give it plenty of specs regarding storage, RAM and CPU (according to Ollama's recommendations)
I chose 32GB and all available cores. Give it plenty of storage to accomodate your models (can be expanded using the GUI)
I chose 60 GB to start. ROCm takes up ~30 GB.

In the LXC container:
install ROCm according to official instructions but do not use DKMS as the drivers are already installed on Proxmox by default.
This is the purpose of...
  • Like
Click to expand...
Hallo @mr44er
Ich kenne den thread bereits aber verstehe es leider nicht mein problem zu lösen, bitte hilf mir dabei was ich genau machen muss, bitte
 
LXC:
Code: 
root@ollama:~# cat /etc/group | grep -w 'render\|\video'
video:x:44:root,ollama
render:x:993:root,ollama


Code: 
root@ollama:~# ls -l /dev/dri/
total 0
drwxr-xr-x 2 root root         80 Sep 18 18:36 by-path
crw-rw---- 1 root video  226,   1 Sep 18 18:36 card1
crw-rw---- 1 root render 226, 128 Sep 18 18:36 renderD128

Pven node:
Code: 
root@pve:~# ls -l /dev/dri
total 0
drwxr-xr-x 2 root root        80 Sep 18 18:36 by-path
crw-rw---- 1 root video 226,   1 Sep 18 18:36 card1
crw-rw---- 1 root   993 226, 128 Sep 18 18:36 renderD128
 
Habe eben eine fehler entdeckt im DevicePass
von /dev/dri/kfd zu /dev/kfd
jetzt bekomme ich bei
rocminfo:

Code: 
root@ollama:~# ls /dev/
console  core  dri  fb0  fd  full  fuse  hugepages  initctl  kfd  log  lxc  mqueue  null  ptmx  pts  random  serial  shm  stderr  stdin  stdout  tty  tty1  tty2  ttyACM0  ttyACM1  ttyUSB0  ttyUSB1  urandom  zero
root@ollama:~# rocminfo
ROCk module is loaded
=====================   
HSA System Attributes   
=====================   
Runtime Version:         1.15
Runtime Ext Version:     1.7
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                             
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
XNACK enabled:           YES
DMAbuf Support:          YES
VMM Support:             NO

==========               
HSA Agents               
==========               
*******                 
Agent 1                 
*******                 
  Name:                                                       
  Uuid:                    CPU-XX                             
  Marketing Name:                                             
  Vendor Name:             CPU                               
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                             
  Node:                    0                                 
  Device Type:             CPU                               
  Cache Info:             
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4308                               
  BDFID:                   0                                 
  Internal Node ID:        0                                 
  Compute Unit:            12                                 
  SIMDs per CU:            0                                 
  Shader Engines:          0                                 
  Shader Arrs. per Eng.:   0                                 
  WatchPts on Addr. Ranges:1                                 
  Memory Properties:       
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED       
      Size:                    48598640(0x2e58e70) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                               
      Alloc Recommended Granule:4KB                               
      Alloc Alignment:         4KB                               
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    48598640(0x2e58e70) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                               
      Alloc Recommended Granule:4KB                               
      Alloc Alignment:         4KB                               
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    48598640(0x2e58e70) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                               
      Alloc Recommended Granule:4KB                               
      Alloc Alignment:         4KB                               
      Accessible by all:       TRUE                               
    Pool 4                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED     
      Size:                    48598640(0x2e58e70) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                               
      Alloc Recommended Granule:4KB                               
      Alloc Alignment:         4KB                               
      Accessible by all:       TRUE                               
  ISA Info:               
*******                 
Agent 2                 
*******                 
  Name:                    gfx90c                             
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Radeon Graphics               
  Vendor Name:             AMD                               
  Feature:                 KERNEL_DISPATCH                   
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                         
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                   
  Queue Type:              MULTI                             
  Node:                    1                                 
  Device Type:             GPU                               
  Cache Info:             
    L1:                      16(0x10) KB                       
    L2:                      1024(0x400) KB                     
  Chip ID:                 5686(0x1636)                       
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1900                               
  BDFID:                   2560                               
  Internal Node ID:        1                                 
  Compute Unit:            7                                 
  SIMDs per CU:            4                                 
  Shader Engines:          1                                 
  Shader Arrs. per Eng.:   1                                 
  WatchPts on Addr. Ranges:4                                 
  Coherent Host Access:    FALSE                             
  Memory Properties:       APU
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                       
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                       
    y                        1024(0x400)                       
    z                        1024(0x400)                       
  Max Waves Per CU:        40(0x28)                           
  Max Work-item Per CU:    2560(0xa00)                       
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 472                               
  SDMA engine uCode::      40                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED     
      Size:                    24299320(0x172c738) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                               
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                               
      Accessible by all:       FALSE                             
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    24299320(0x172c738) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                               
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                               
      Accessible by all:       FALSE                             
    Pool 3                   
      Segment:                 GROUP                             
      Size:                    64(0x40) KB                       
      Allocatable:             FALSE                             
      Alloc Granule:           0KB                               
      Alloc Recommended Granule:0KB                               
      Alloc Alignment:         0KB                               
      Accessible by all:       FALSE                             
  ISA Info:               
    ISA 1                   
      Name:                    amdgcn-amd-amdhsa--gfx90c:xnack+   
      Machine Models:          HSA_MACHINE_MODEL_LARGE           
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                       
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                       
        y                        1024(0x400)                       
        z                        1024(0x400)                       
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
    ISA 2                   
      Name:                    amdgcn-amd-amdhsa--gfx9-generic:xnack+
      Machine Models:          HSA_MACHINE_MODEL_LARGE           
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                       
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                       
        y                        1024(0x400)                       
        z                        1024(0x400)                       
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*** Done ***             
root@ollama:~# ollama run mistral
>>>
Use Ctrl + d or /bye to exit.
>>>
root@ollama:~# ollama ps
NAME              ID              SIZE      PROCESSOR    CONTEXT    UNTIL             
mistral:latest    6577803aa9a0    5.3 GB    100% CPU     4096       4 minutes from now   
root@ollama:~# radeontop
Collecting data, please wait....
root@ollama:~#


jedoch immer noch cpu als processor

und jetzt bekomme ich garkeine antwort mehr vom LLM
 
Keugon said:
Habe eben eine fehler entdeckt im DevicePass
von /dev/dri/kfd zu /dev/kfd
jetzt bekomme ich bei
rocminfo:
Click to expand...
Gut, wenn rocminfo was ausgibt, passt das. Das GPU mit dem LXC teilen geht immer nach dem selben Schema, bei ollama kann ich jetzt aber nicht helfen, das nutze ich nicht.

Diese helper-scripte allerdings auch nicht.
 
zur Info:
Mit rocm 7.1 hat dann alles Technisch funktioniert jedoch wird die integriegrte gpu beim survey ignoriert weil sie nur "0.5gb " vram hat.
Ich muss das noch Testen ob es mit Bios ram zuweisung dann vollständig geht.
 
You must log in or register to reply here.
Top Bottom