Proxmox + Older NVIDIA GPUs (GTX 1060 / P4000 / V100) LXC Passthrough Guide (Updated June 2026)

I

itamarbudin

New Member
Aug 13, 2024
9
1
3

Recommended Configurations​

Most Proven Configuration​

Code: 
Proxmox Kernel 6.14 / 6.17
NVIDIA Driver 575.57.08
CUDA 12.9.1

Works well with:

  • GTX 1060 / 1070 / 1080
  • Quadro P4000 / P5000
  • Tesla V100

New Configuration (Kernel 7.x)​

Code: 
Proxmox Kernel 7.x
NVIDIA Driver 580.159.04
CUDA 13.x

Reported working with:

  • Tesla V100
  • Various Volta-based GPUs
Kernel 7.x support is still evolving for older Pascal cards. If stability is your top priority, stay on Kernel 6.x.

HOST INSTALLATION​

Install build tools:

Code: 
apt update
apt install -y build-essential dkms pve-headers-$(uname -r)

Download NVIDIA driver:

575 branch:

Code: 
wget https://us.download.nvidia.com/tesla/575.57.08/NVIDIA-Linux-x86_64-575.57.08.run

580 branch:

Code: 
wget https://us.download.nvidia.com/tesla/580.159.04/NVIDIA-Linux-x86_64-580.159.04.run

Make executable:

Code: 
chmod +x NVIDIA-Linux-x86_64-580.159.04.run

Install:

Code: 
./NVIDIA-Linux-x86_64-580.159.04.run

Installer recommendations:

  • YES to DKMS
  • NO to OpenGL libraries (headless server)
Reboot:

Code: 
reboot

Verify:

Code: 
nvidia-smi

LXC GPU PASSTHROUGH​

Edit container config:

Code: 
nano /etc/pve/lxc/.conf

Add:

Code: 
lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 509:* rwm
lxc.cgroup2.devices.allow: c 511:* rwm

lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

Restart:

Code: 
pct restart

MULTIPLE GPU SETUPS​

Check GPU numbering:

Code: 
nvidia-smi -L

Example:

Code: 
GPU 0: Tesla V100
GPU 1: Quadro P4000
GPU 2: GTX 1060

Pass only a specific GPU:

Code: 
lxc.mount.entry: /dev/nvidia1 dev/nvidia1 none bind,optional,create=file

Pass multiple GPUs:

Code: 
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidia2 dev/nvidia2 none bind,optional,create=file

Limit CUDA visibility:

Code: 
export CUDA_VISIBLE_DEVICES=0

or

Code: 
export CUDA_VISIBLE_DEVICES=0,1

INSIDE THE CONTAINER​

Download the same driver version as the host.

Example:

Code: 
wget https://us.download.nvidia.com/tesla/580.159.04/NVIDIA-Linux-x86_64-580.159.04.run

Install userspace libraries only:

Code: 
chmod +x NVIDIA-Linux-x86_64-580.159.04.run

./NVIDIA-Linux-x86_64-580.159.04.run --no-kernel-module

Verify:

Code: 
nvidia-smi

OPTIONAL CUDA INSTALLATION​

CUDA 12.9.1 (575 branch):

Code: 
wget https://developer.download.nvidia.com/compute/cuda/12.9.1/local_installers/cuda_12.9.1_575.57.08_linux.run

Install:

Code: 
chmod +x cuda_12.9.1_575.57.08_linux.run

./cuda_12.9.1_575.57.08_linux.run

Skip driver installation during CUDA setup.

CURRENT RECOMMENDATION​

For Tesla V100:
Code: 
Kernel 7.x + 580.159.04

For mixed environments (GTX 1060 + P4000 + V100):
Code: 
Kernel 6.14/6.17 + 575.57.08

until broader Kernel 7.x compatibility is consistently confirmed across Pascal GPUs.
 
Last edited:
580.159.04 is the latest driver which support nvidia v100, and DKMS compile failures on kernel 7 seems has been fixed in this version
 
  • Like
Reactions: itamarbudin
imlk said:
580.159.04 is the latest driver which support nvidia v100, and DKMS compile failures on kernel 7 seems has been fixed in this version
Click to expand...
Thank you. I tried this version and can confirm it works with kernels 6.17 and 7.x. I will update the guide
 
Thanks for this guide. I just got my Quadro P4000 working. It didn't like kernel 7.x or 6.17, but 6.14 worked like a charm. NVIDIA Driver 575.57.08 installed, and the device was able to pass through to my jellyfin container successfully
 
sphogdrois said:
Thanks for this guide. I just got my Quadro P4000 working. It didn't like kernel 7.x or 6.17, but 6.14 worked like a charm. NVIDIA Driver 575.57.08 installed, and the device was able to pass through to my jellyfin container successfully
Click to expand...
You're welcome :)

The problem is that when you are using a passthrough, the kernel driver might be blocking you from installing the driver.

see here: https://forum.proxmox.com/threads/unable-to-load-the-kernel-module-nvidia-ko.89428/
 
Update: Proxmox VE 9.2, IOMMU, LXC GPU sharing, and VFIO testing

I completed some additional testing and found several details worth adding to the guide.

Tested configuration

  • Proxmox VE 9.2.4
  • Kernel 7.0.14-6-pve
  • AMD EPYC 7352
  • ASRock Rack ROMED8-2T
  • NVIDIA driver 580.159.04
  • Two Tesla V100 SXM2 32 GB GPUs shared with an unprivileged LXC
  • One Quadro RTX 5000 passed exclusively to an Ubuntu desktop VM

Important: LXC GPU sharing and VM passthrough are different

There are two separate configurations:

  1. LXC device sharing

    The GPU remains bound to the host NVIDIA driver. Selected /dev/nvidia* devices are passed into the container, and the container uses the host kernel driver.

    IOMMU/VFIO is normally not required for this method.
  2. Full PCI passthrough to a VM

    The GPU is detached from the host NVIDIA driver and bound to vfio-pci. The VM then owns the complete PCI device.

    IOMMU is required for this method.

BIOS settings used for VFIO

On the ROMED8-2T, I enabled:

  • SVM
  • IOMMU
  • Above 4G Decoding

SR-IOV and ARI may be useful on some systems, but they are not universally required for conventional GPU passthrough.

Before enabling IOMMU, the host had:

  • No ACPI IVRS table
  • No devices under /sys/class/iommu
  • No IOMMU groups
  • No iommu_group link for the GPU
  • A kernel message showing PCI-DMA using SWIOTLB

Useful checks:

Bash: 
journalctl -k -b | grep -Ei 'AMD-Vi|IVRS|IOMMU|PCI-DMA'
ls -la /sys/class/iommu
ls -la /sys/kernel/iommu_groups
ls -l /sys/bus/pci/devices/0000:81:00.0/iommu_group

Correction to the LXC configuration commands

The container configuration path must include the CT ID:

Bash: 
nano /etc/pve/lxc/<CTID>.conf

Restart or start the container with its ID:

Bash: 
pct restart <CTID>

or:

Bash: 
pct start <CTID>

Proxmox VE 9 supports managed device entries such as:

Code: 
dev0: /dev/nvidia0,gid=44
dev1: /dev/nvidiactl,gid=44
dev2: /dev/nvidia-uvm,gid=44
dev3: /dev/nvidia-uvm-tools,gid=44

Additional GPUs and required /dev/nvidia-caps devices can be added as separate devN entries.

The correct group ID, permissions, device majors, device minors, and cap devices must be checked on the actual host. Values should not be copied blindly from another installation.

Run nvidia-smi before starting the container

After assigning the RTX 5000 to VFIO, my LXC initially failed with:

Code: 
Device /dev/nvidia1 does not exist

On this host, running nvidia-smi once initialized the NVIDIA device nodes:

Bash: 
nvidia-smi
ls -la /dev/nvidia*
pct start <CTID>

After that, the container started and both V100 GPUs worked correctly.

This may not affect every system, but it is a useful preflight check after boot, driver reload, or changing which GPUs are assigned to VFIO.

Do not assume the nvidia-smi index matches /dev/nvidiaN

The GPU index displayed by nvidia-smi did not match the NVIDIA device minor on my system.

Check the actual PCI-address-to-device-minor mapping:

Bash: 
for gpu in /proc/driver/nvidia/gpus/*; do
    echo "=== ${gpu##*/} ==="
    grep "Device Minor" "$gpu/information"
done

In my test:

Code: 
PCI 47:00.0 -> Device Minor 1
PCI 82:00.0 -> Device Minor 0

The LXC configuration must use the actual device nodes present under /dev, not assumptions based only on the order shown by nvidia-smi.

Multifunction GPU passthrough

The Quadro RTX 5000 exposed four PCI functions:

Code: 
81:00.0 VGA controller
81:00.1 Audio device
81:00.2 USB controller
81:00.3 USB Type-C/UCSI controller

For complete VM passthrough, I bound all four functions to vfio-pci.

Verification:

Bash: 
for pci in 81:00.0 81:00.1 81:00.2 81:00.3; do
    echo "=== $pci ==="
    lspci -nnk -s "$pci"
done

Each function should show:

Code: 
Kernel driver in use: vfio-pci

Expected host nvidia-smi behavior

A GPU bound to vfio-pci will not appear in the host’s nvidia-smi output. That is expected because the host NVIDIA driver no longer owns it.

In the final configuration:

  • The two V100 GPUs remained visible to the Proxmox host and the LXC.
  • The RTX 5000 disappeared from the host’s nvidia-smi.
  • The RTX 5000 was visible and working inside the Ubuntu VM.

This mixed arrangement is working successfully:

  • V100 GPUs: host NVIDIA driver → shared with LXC
  • RTX 5000: VFIO → dedicated desktop VM

Proxmox documentation:

pct container-device documentation

Proxmox NVIDIA/VFIO preparation guidance
 
You must log in or register to reply here.
Top Bottom
Back