Upgrade problem to 8.2 (6.8 kernel - nvidia drivers)

Hi,
Before Kernel 6.8 I used nvidia-driver package from Debian and all worked fine. I use VirtGL driver for VM and it was able to use HW for render.
After updating to Kernel 6.8, I removed nvidia-driver package and installed Driver ver 550 using the NVidia script but it does not work as expected.

VirtGL does not use GPU but CPU for GL rendering on VM, also before installing new driver, using nvidia-smi I saw some process that uses GPU, after installing NVidia 550 driver, GPU is not used.

I suspect that the Debian package does some configuration like installing GL libs and sets alternatives to use NVIdia HW driver for GL as during installation of nvidia-driver, lots of other packages are installed.

I am not familiar with the Linux OpenGL/Mesa subsystem to do this manually. So is there any tutorial how to setup everything step by step and do the same configuration as is done during the installation of nvidia-driver form the Debian package?
 
dooferorg said:
Sucks too looking at all the hardware now dropped from the v17 branch
Click to expand...
I have the P4 as well, and I believe it is still LTS in 16.X until next year. I know 17 just got an update of some sort on the not "v" gpu side, so hopefully that will trickle down.
 
macboy80 said:
I have the P4 as well, and I believe it is still LTS in 16.X until next year. I know 17 just got an update of some sort on the not "v" gpu side, so hopefully that will trickle down.
Click to expand...
Let's hope so as I rely on the 16.X host drivers for a Xpenology VM which still uses 15.X guest drivers.
 
Urbaman said:
Hi @dooferorg

I went through the new patch for 535.161.05, the driver compiles, I see the GPU in nvidia-smi, but nothing shows in mdevctl types.

The nodes still on 535.129.03, kernel 6.5 work flawlessly.

I hope my P2200 didn't get completely dropped (it gets seen as P40 from guests in 6.5 kernel nodes)
Click to expand...
Sorry you're having issue with it. I did wonder if the patch that's on the website indicated is giving issues - it was before and I got the patch file directly from the author via Discord - I thought maybe the gitlab problem was something I did.
I attached the file I used to patch the 16.5 driver in this post: https://forum.proxmox.com/threads/pve-8-22-kernel-6-8-and-nvidia-vgpu.147039/

Now it should be mentioned too that the patch is in addition to the polloloco patching needing for adding support for older cards as well.
 
Hi all !

[PVE 8.2]
JUST A LITTLE TIP TO COMPILE DEBIAN 'NVIDIA-DRIVER' DKMS MODULES AGAINST 6.8* NEW KERNELS (which didn't compile, causing kernel pinning)

[i'm french, so, please excuse my somewhat poor english ! ;) ]

For those who have problems installing new 6.8.* linux kernel (which don't compile DEBIAN nvidia-driver against greater version than 6.7.*) with DKMS etc. I've a good news ! ;)

A french guy (here : <https://forums.fedora-fr.org/d/74709-après-mise-à-jour-du-noyau-67-vers-68-carte-nvidia-plus-reconnue/19>) found a simple workaround to pass these issues.
Details are on the site, but if I summarize, the guy saw in the « failed compilation log » (/var/lib/dkms/nvidia-current/<your driver version>/build/make.log) an error message about : "error: ‘DRM_UNLOCKED’ undeclared here (not in a function); did you mean ‘VM_LOCKED’?".

This is related on missing code in newer versions (from 6.8) of kernel in file 'drm_ioctl.h'.
This file is in /usr/src/linux-headers-<your_kernel_version>-pve/include/drm/ or in /usr/src/linux-image... ; just do a 'locate' or 'dlocate' + grep to find where it is on your system.
At the beginning, the guy explained what lines editing (adding) in the new file, copying from file of the older versions (pre-6.8*) -- here, I took it from /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h, but it's OK with 6.7 also -- BUT, one don't need to bother with editing, as everyone can see here, there's nothing else than the required code in the file :
diff -B /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h /usr/src/linux-headers-6.8.4-3-pve/include/drm/drm_ioctl.h
113,123d112
< * @DRM_UNLOCKED:
< *
< * Whether &drm_ioctl_desc.func should be called with the DRM BKL held
< * or not. Enforced as the default for all modern drivers, hence there
< * should never be a need to set this flag.
< *
< * Do not use anywhere else than for the VBLANK_WAIT IOCTL, which is the
< * only legacy IOCTL which needs this.
< */
< DRM_UNLOCKED = BIT(4),
< /**

So, just replacing current drm_ioctl.h file by older one works fine !

If you're fear of doing it from scratch, just do a backup of the file before replacing it, but it's not really necessary if the diff confirms that only the code above differs in the two files.

Once the change is done you'd happily see this in 'apt install':
[sorry, it's FR logs, coz I'm french]

Paramétrage de proxmox-kernel-6.8.4-3-pve-signed (6.8.4-3) ...
[...]
Building module:
Cleaning build area...
[...]
Signing module /var/lib/dkms/nvidia-current/525.147.05/build/nvidia-peermem.ko
Cleaning build area...

nvidia-current.ko:
Running module version sanity check.
[...]
nvidia-current-modeset.ko:
Running module version sanity check.
[...]
nvidia-current-drm.ko:
Running module version sanity check.
[...]
nvidia-current-uvm.ko:
Running module version sanity check.
[...]
nvidia-current-peermem.ko:
Running module version sanity check.
[...]
- Installing to /lib/modules/6.8.4-3-pve/updates/dkms/
depmod...
[...]
uilding module:
Cleaning build area...
env NV_VERBOSE=1 make -j16 modules KERNEL_UNAME=6.8.4-3-pve............
Signing module /var/lib/dkms/nvidia-current-open/525.147.05/build/nvidia.ko
[...]
dkms: autoinstall for kernel: 6.8.4-3-pve.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 6.8.4-3-pve /boot/vmlinuz-6.8.4-3-pve
update-initramfs: Generating /boot/initrd.img-6.8.4-3-pve
[...]
Generating grub configuration file ...
[...]
done
Progression : [ 91%]

And it's all done ! ;)

Very simple, isn't it ?
Hoping we'll may apply this "sort of patching" to all new kernels till debian / PVE devs change these few lines in 'drm_ioctl.h' file.

My 0.2€ ;)
 
Last edited:
  • Like
Reactions: Backroads4me, biotuxic and Yogorov
jeep said:
Hi all !

[PVE 8.2]
JUST A LITTLE TIP TO COMPILE DEBIAN 'NVIDIA-DRIVER' DKMS MODULES AGAINST 6.8* NEW KERNELS (which didn't compile, causing kernel pinning)

......
Click to expand...


Thanks for your post! it Really save my day . solved my problem.

base on your post here is my summary:
linux kernel 6.7.* above won't fit for Debian nvidia-driver and DKMS
main blame on this file drm_ioctl.h

solution:
install old version kernel source code and take this file drm_ioctl.h for new kernel

Command:
apt install pve-headers-6.5.13-5-pve
cp /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h /usr/src/linux-headers-6.8.4-3-pve/include/drm/drm_ioctl.h
apt -f install
apt install nvidia-driver
 
  • Like
Reactions: mariol and Backroads4me
Yogorov said:
Thanks for your post! it Really save my day . solved my problem.
Click to expand...
Nice to read, Yogorov ; it was the goal of my post, really happy if it does the job.
Yogorov said:
base on your post here is my summary:
linux kernel 6.7.* above won't fit for Debian nvidia-driver and DKMS
main blame on this file drm_ioctl.h

solution:
install old version kernel source code and take this file drm_ioctl.h for new kernel

Command:
apt install pve-headers-6.5.13-5-pve
cp /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h /usr/src/linux-headers-6.8.4-3-pve/include/drm/drm_ioctl.h
apt -f install
apt install nvidia-driver
Click to expand...
Perfect summary IMHO, for those who don't have time to read a lot, it's quite good, thx.
[but, note : if your nvidia-driver is already the latest version ('apt-cache policy nvidia-driver'), don't need to install again, so
the last line isn't needed, just fetch the sources (or the headers) and copy the old "drm_ioctl.h" version in place of the
new one, then do an 'apt upgrade' (or 'apt-get -V --with-new-pkgs --show-progress upgrade', or 'apt install', or...) and... voilà ;)]
 
Last edited:
jeep said:
Nice to read, Yogorov ; it was the goal of my post, really happy if it does the job.

Perfect summary IMHO, for those who don't have time to read a lot, it's quite good, thx.
[but, note : if your nvidia-driver is already the latest version ('apt-cache policy nvidia-driver'), don't need to install again, so
the last two lines aren't needed, just fetch the sources (or the headers) and copy the old "drm_ioctl.h" version in place of the
new one.]
Click to expand...

yeah, last two line command may not need
because my summary was to short and lack too much information
last two line command may solve this problem at all
 
Yogorov said:
yeah, last two line command may not need
because my summary was to short and lack too much information
last two line command may solve this problem at all
Click to expand...
Yep ! Understand.
From my side, I realized I made a little mistake (i just rectified/edit), just one line is not needed, the last one (or another, see my post above) is really needed, coz' you'll have to compile the dkms modules against the "brand new hacked kernel", in order to be up-to-date.
Then -- maybe, if you pinned kernel -- a 'proxmox-boot-tool kernel unpin <your_pinned_kernel>' in order to boot the brand new kernel.
Here, it was : 'proxmox-boot-tool kernel unpin 6.5.13-5-pve'.
 
Last edited:
  • Like
Reactions: Yogorov
jeep said:
Hi all !

[PVE 8.2]
JUST A LITTLE TIP TO COMPILE DEBIAN 'NVIDIA-DRIVER' DKMS MODULES AGAINST 6.8* NEW KERNELS (which didn't compile, causing kernel pinning)

[i'm french, so, please excuse my somewhat poor english ! ;) ]

For those who have problems installing new 6.8.* linux kernel (which don't compile DEBIAN nvidia-driver against greater version than 6.7.*) with DKMS etc. I've a good news ! ;)

A french guy (here : <https://forums.fedora-fr.org/d/74709-après-mise-à-jour-du-noyau-67-vers-68-carte-nvidia-plus-reconnue/19>) found a simple workaround to pass these issues.
Details are on the site, but if I summarize, the guy saw in the « failed compilation log » (/var/lib/dkms/nvidia-current/<your driver version>/build/make.log) an error message about : "error: ‘DRM_UNLOCKED’ undeclared here (not in a function); did you mean ‘VM_LOCKED’?".

This is related on missing code in newer versions (from 6.8) of kernel in file 'drm_ioctl.h'.
This file is in /usr/src/linux-headers-<your_kernel_version>-pve/include/drm/ or in /usr/src/linux-image... ; just do a 'locate' or 'dlocate' + grep to find where it is on your system.
At the beginning, the guy explained what lines editing (adding) in the new file, copying from file of the older versions (pre-6.8*) -- here, I took it from /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h, but it's OK with 6.7 also -- BUT, one don't need to bother with editing, as everyone can see here, there's nothing else than the required code in the file :
diff -B /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h /usr/src/linux-headers-6.8.4-3-pve/include/drm/drm_ioctl.h
113,123d112
< * @DRM_UNLOCKED:
< *
< * Whether &drm_ioctl_desc.func should be called with the DRM BKL held
< * or not. Enforced as the default for all modern drivers, hence there
< * should never be a need to set this flag.
< *
< * Do not use anywhere else than for the VBLANK_WAIT IOCTL, which is the
< * only legacy IOCTL which needs this.
< */
< DRM_UNLOCKED = BIT(4),
< /**

So, just replacing current drm_ioctl.h file by older one works fine !

If you're fear of doing it from scratch, just do a backup of the file before replacing it, but it's not really necessary if the diff confirms that only the code above differs in the two files.

Once the change is done you'd happily see this in 'apt install':
[sorry, it's FR logs, coz I'm french]

Paramétrage de proxmox-kernel-6.8.4-3-pve-signed (6.8.4-3) ...
[...]
Building module:
Cleaning build area...
[...]
Signing module /var/lib/dkms/nvidia-current/525.147.05/build/nvidia-peermem.ko
Cleaning build area...

nvidia-current.ko:
Running module version sanity check.
[...]
nvidia-current-modeset.ko:
Running module version sanity check.
[...]
nvidia-current-drm.ko:
Running module version sanity check.
[...]
nvidia-current-uvm.ko:
Running module version sanity check.
[...]
nvidia-current-peermem.ko:
Running module version sanity check.
[...]
- Installing to /lib/modules/6.8.4-3-pve/updates/dkms/
depmod...
[...]
uilding module:
Cleaning build area...
env NV_VERBOSE=1 make -j16 modules KERNEL_UNAME=6.8.4-3-pve............
Signing module /var/lib/dkms/nvidia-current-open/525.147.05/build/nvidia.ko
[...]
dkms: autoinstall for kernel: 6.8.4-3-pve.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 6.8.4-3-pve /boot/vmlinuz-6.8.4-3-pve
update-initramfs: Generating /boot/initrd.img-6.8.4-3-pve
[...]
Generating grub configuration file ...
[...]
done
Progression : [ 91%]

And it's all done ! ;)

Very simple, isn't it ?
Hoping we'll may apply this "sort of patching" to all new kernels till debian / PVE devs change these few lines in 'drm_ioctl.h' file.

My 0.2€ ;)
Click to expand...
It works great. Thanks for the trick
 
Yogorov said:
Command:
apt install pve-headers-6.5.13-5-pve
cp /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h /usr/src/linux-headers-6.8.4-3-pve/include/drm/drm_ioctl.h
apt -f install
apt install nvidia-driver
Click to expand...

Here is the full command list which worked in my case:

Bash: 
sudo apt update

apt install -y pve-headers-$(uname -r)
apt install build-essential pkg-config xorg xorg-dev libglvnd0 libglvnd-dev
apt install -y dkms pve-headers wget

apt install pve-headers-6.5.13-5-pve
cp /usr/src/linux-headers-6.5.13-5-pve/include/drm/drm_ioctl.h /usr/src/linux-headers-$(uname -r)/include/drm/drm_ioctl.h

cd ~
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/535.104.05/NVIDIA-Linux-x86_64-535.104.05.run
chmod +x NVIDIA-Linux-x86_64-535.104.05.run

./NVIDIA-Linux-x86_64-535.104.05.run --dkms
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back