vGPU VFs with Proxmox 8 and Plex LXC not working for HDR tone mapping

T

TrustyHippo

New Member
Apr 9, 2023
18
8
3
I have an Intel i5-1240P host with Proxmox 8 that has been configured for vGPU via DKMS. I have 7 vGPU VFs. I've assigned one to a Windows 11 VM, and it's working perfectly with the Intel drivers. However, my Plex LXC (tteck script) is having issues with HDR tone mapping (which uses the GPU). Transcoding works fine, but that uses the Intel XE graphics engine (different API) than tone mapping which directly uses the GPU.

I should add this same LXC worked fine with HDR tone mapping and transcoding with Proxmox 7.4 using the 6.2 kernel. It's only when trying to use the VF vGPU does tone mapping fail.

Inside the Plex LXC without any LXC config file changes I see:

Code: 
drw-rw---- 2 root root        320 Jun 25 14:26 by-path
crw-rw---- 1 root video  226,   0 Jun 25 13:33 card0
crw-rw---- 1 root video  226,   2 Jun 25 13:33 card2
crw-rw---- 1 root video  226,   3 Jun 25 13:33 card3
crw-rw---- 1 root video  226,   4 Jun 25 13:33 card4
crw-rw---- 1 root video  226,   5 Jun 25 13:33 card5
crw-rw---- 1 root video  226,   6 Jun 25 13:33 card6
crw-rw---- 1 root video  226,   7 Jun 25 13:33 card7
crw-rw---- 1 root render 226, 128 Jun 25 13:33 renderD128
crw-rw---- 1 root render 226, 130 Jun 25 13:33 renderD130
crw-rw---- 1 root render 226, 131 Jun 25 13:33 renderD131
crw-rw---- 1 root render 226, 132 Jun 25 13:33 renderD132
crw-rw---- 1 root render 226, 133 Jun 25 13:33 renderD133
crw-rw---- 1 root render 226, 134 Jun 25 13:33 renderD134
crw-rw---- 1 root render 226, 135 Jun 25 13:33 renderD135

With my modified LXC:

Code: 
root@plex:/dev/dri# ls -l
total 0
crw-rw---- 1 root video  226,   2 Jun 26 10:29 card0
crw-rw---- 1 root render 226, 130 Jun 26 10:29 renderD128


This is the original LXC config:

Code: 
## Plex LXC
#  ### https://tteck.github.io/Proxmox/
#  <a href='https://ko-fi.com/D1D7EP4GF'><img src='https://img.shields.io/badge/%E2%98%95-Buy me a coffee-red' /></a>
arch: amd64
cores: 2
features: nesting=1
hostname: plex
memory: 2048
net0: name=eth0,bridge=vmbr0,hwaddr=D6:E9:00:12:38:78,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-102-disk-0,size=8G
swap: 512
tags: proxmox-helper-scripts
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file

This is the modified LXC config:

Code: 
## Plex LXC
#  ### https://tteck.github.io/Proxmox/
#  <a href='https://ko-fi.com/D1D7EP4GF'><img src='https://img.shields.io/badge/%E2%98%95-Buy me a coffee-red' /></a>
#lxc.cgroup2.devices.allow%3A c 29%3A0 rwm
#lxc.mount.entry%3A /dev/fb0 dev/fb0 none bind,optional,create=file
arch: amd64
cores: 2
features: nesting=1
hostname: plex
memory: 768
nameserver: 10.13.2.1
net0: name=eth0,bridge=vmbr0,gw=10.13.2.1,hwaddr=5A:11:2E:E7:67:4B,ip=10.13.2.212/24,type=veth
onboot: 0
ostype: ubuntu
rootfs: local-lvm:vm-109-disk-1,size=20G
swap: 128
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.cgroup2.devices.allow: c 226:2 rwm
lxc.cgroup2.devices.allow: c 226:130 rwm
lxc.mount.entry: /dev/dri/card2 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD130 dev/dri/renderD128 none bind,optional,create=file
 
Last edited:
  • Like
Reactions: semanticbeeng
I did a bit more troubleshooting. If I enable vGPU and via the LXC pass through ONLY the primary VGA device then tone mapping works.

Code: 
## Plex LXC
#  ### https://tteck.github.io/Proxmox/
#  <a href='https://ko-fi.com/D1D7EP4GF'><img src='https://img.shields.io/badge/%E2%98%95-Buy me a coffee-red' /></a>
#lxc.cgroup2.devices.allow%3A c 29%3A0 rwm
#lxc.mount.entry%3A /dev/fb0 dev/fb0 none bind,optional,create=file
arch: amd64
cores: 2
features: nesting=1
hostname: plex
memory: 2048
net0: name=eth0,bridge=vmbr0,hwaddr=4A:8E:41:2F:17:8B,ip=dhcp,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-lvm:vm-102-disk-0,size=8G
swap: 512
tags: proxmox-helper-scripts
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file

However, if I use any VF then tone mapping fails but transcoding still works:

Code: 
lxc.cgroup2.devices.allow: c 226:4 rwm
lxc.cgroup2.devices.allow: c 226:132 rwm
lxc.mount.entry: /dev/dri/card4 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD132 dev/dri/renderD128 none bind,optional,create=file
 
TrustyHippo said:
I did a bit more troubleshooting. If I enable vGPU and via the LXC pass through ONLY the primary VGA device then tone mapping works.

Code: 
## Plex LXC
#  ### https://tteck.github.io/Proxmox/
#  <a href='https://ko-fi.com/D1D7EP4GF'><img src='https://img.shields.io/badge/%E2%98%95-Buy me a coffee-red' /></a>
#lxc.cgroup2.devices.allow%3A c 29%3A0 rwm
#lxc.mount.entry%3A /dev/fb0 dev/fb0 none bind,optional,create=file
arch: amd64
cores: 2
features: nesting=1
hostname: plex
memory: 2048
net0: name=eth0,bridge=vmbr0,hwaddr=4A:8E:41:2F:17:8B,ip=dhcp,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-lvm:vm-102-disk-0,size=8G
swap: 512
tags: proxmox-helper-scripts
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file

However, if I use any VF then tone mapping fails but transcoding still works:

Code: 
lxc.cgroup2.devices.allow: c 226:4 rwm
lxc.cgroup2.devices.allow: c 226:132 rwm
lxc.mount.entry: /dev/dri/card4 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD132 dev/dri/renderD128 none bind,optional,create=file
Click to expand...

Maybe you should also post the issue here i think its not a proxmox issue but more the drivers woth sriov support we use https://github.com/strongtz/i915-sriov-dkms?ref=michaels-tinkerings
 
You could try installing the github sriov drivers in the lxc because we updated the drivers on the host but not in the lxc i thought that was not necessary but maybe it is. the lxc uses the original drivers that do not support sriov idk its just an idea
 
pbvdven said:
You could try installing the github sriov drivers in the lxc because we updated the drivers on the host but not in the lxc i thought that was not necessary but maybe it is. the lxc uses the original drivers that do not support sriov idk its just an idea
Click to expand...
Because when you use a debian vm you also need to install the sriov supported drivers in the guest to make it work at all.
 
pbvdven said:
You could try installing the github sriov drivers in the lxc because we updated the drivers on the host but not in the lxc i thought that was not necessary but maybe it is. the lxc uses the original drivers that do not support sriov idk its just an idea
Click to expand...
I'm terrible with trying to get video drivers on Linux installed...is there a step by step procedure I can use in the LXC to validate?
 
TrustyHippo said:
I'm terrible with trying to get video drivers on Linux installed...is there a step by step procedure I can use in the LXC to validate?
Click to expand...
I’ll try tomorrow i now got some errors because of mismatch kernel headers in the lxc and i need to go to sleep now. I know there is an debian explanation in the github maybe that. Could give some guidance use debian bookworm lxc for testing because its already has headers 6.1 then you could skip that part. Its basically the same as installing on the pve host. I give it another try tomorrow ill keep you updated if i have any luck
 
He mentioned on the readme

We will need to run the same driver under Linux guests. We can repeat the steps for installing the driver. However, when modifying command line defaults, we use i915.enable_guc=3 instead of i915.enable_guc=3 i915.max_vfs=7. Furthermore, we don't need to use sysfsutils to create any more VFs since we ARE using a VF. Once that's done, update grub and initramfs, then reboot. Once the VM is back up again, do dmesg | grep i915 to see if your VF is recognized by the kernel. Optionally, install vainfo, then do vainfo to see if the iGPU has been picked up by the VAAPI.
 
pbvdven said:
He mentioned on the readme

We will need to run the same driver under Linux guests. We can repeat the steps for installing the driver. However, when modifying command line defaults, we use i915.enable_guc=3 instead of i915.enable_guc=3 i915.max_vfs=7. Furthermore, we don't need to use sysfsutils to create any more VFs since we ARE using a VF. Once that's done, update grub and initramfs, then reboot. Once the VM is back up again, do dmesg | grep i915 to see if your VF is recognized by the kernel. Optionally, install vainfo, then do vainfo to see if the iGPU has been picked up by the VAAPI.
Click to expand...
But an lxc is a bit different it doesn’t have grub default commandline but he states that lxc should also work but not really explaining how or ive you need to install drivers. But i think so because the lxc has its own drivers and those dont have the sriov support at the moment so you need to install the github drivers in the lxc but im getting kernel header errors so for now im done.
 
pbvdven said:
I got it working but its not working i got all green lines but its with hdr active and still (hw) i used a debian bookworm lxc updated the i915 drivers and installed plex but now i have the same issue as https://github.com/strongtz/i915-sriov-dkms/issues/57
Click to expand...

Yes I have no issues getting Plex to recognize the GPU, but enabling HDR tone mapping results in a corrupted video stream. Green, torn image, etc.
 
TrustyHippo said:
Yes I have no issues getting Plex to recognize the GPU,
Click to expand...
How? what did you do?
I have tried emby and vainfo and they both get a -1 error
also the emby log implies that the container is seeing *all* of the 7 VFs - so not sure how the remapping is supposed to work.
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back