How to use one GPU as passthrough and another identical GPU in LXCs?

8192K

Member
Apr 12, 2024
55
6
8
I have two exactly identical GPUs (Nvidia). I'd like to pass one through to a VM and use the other one in several LXCs. In order to use passthrough, I have to blacklist the nvidia/nouveau drivers rendering the LXC one unusable.

How can I achieve this?
 
With Radeon, you don't need to add it to the blacklist if you unbind the driver, but is that not possible with NVIDIA?

I don't have one, so I can't test it or provide guidance.
 
Can you be more precise, please? Radeon passthrough works without adding the driver to the blacklist? I don't think so, honestly, as I had a Radeon in there until recently.
What exactly do you mean by "unbind"?
 
What exactly do you mean by "unbind"?

Code:
echo "0000:04:00.0" > /sys/bus/pci/drivers/"driver name"/unbind 2>/dev/null

This refers to using a command to remove the driver “driver name” bound to pci “0000:04:00.0”.

A blacklist prevents the driver from being used from the start, whereas a command initially allows the device to operate with the driver. The command is then executed to remove the driver from a specific PCI device afterward.

Therefore, you can remove the driver from a specific device without adding it to the blacklist.


0000:04:00.0 and amdgpu according to your environment.

*amdgpu is the driver name.
*The value to be specified can be confirmed using lspci -nnk.

If audio devices require similar processing, you can add that line.

Code:
mkdir -p /var/lib/vz/snippets
nano /var/lib/vz/snippets/gpu_reset.sh
---
#!/bin/bash

phase="$2"
echo "Phase is $phase"
if [ "$phase" == "pre-start" ]; then
echo "0000:04:00.0" > /sys/bus/pci/drivers/amdgpu/unbind 2>/dev/null
sleep 2
echo 8 > /sys/bus/pci/devices/0000:04:00.0/resource2_resize
sleep 2
elif [ "$phase" == "post-stop" ]; then
sleep 5
echo "0000:04:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind 2>/dev/null
sleep 2
echo "0000:04:00.0" > /sys/bus/pci/drivers/amdgpu/bind 2>/dev/null
sleep 2
fi
---
chmod +x /var/lib/vz/snippets/gpu_reset.
qm set VMID -hookscript local:snippets/gpu_reset.sh

All settings added for the following PCI passthrough must be removed.

blacklist XXX
/etc/modprobe.d/vfio.conf
grub kernel parameters

Radeon passthrough works without adding the driver to the blacklist?

I'm running this script on the following hardware:

Radeon RX 9070 XT
Ryzen integrated GPU
Intel integrated GPU

Therefore, I don't think NVIDIA can't do the same thing.

I don't think so, honestly, as I had a Radeon in there until recently

The person who created the script in the first thread, as well as those who posted after the aforementioned thread, are functioning similarly.
 
Last edited: