GPU passthrough to Plex VM - nvidia-smi: No devices were found

gyik · Mar 15, 2022

Hello,

Running Proxmox 7.1-10 on an Asrockrack X570D4I-2T with a 3700X.
Plex VM is an Ubuntu 20.04.4 LTS. Just grabbed an NV Quadro P400 and wanted to passthrough to do the transcoding. Followed Craft Computer's guide link, but the very last step is not working for me.
This can be found here 21:50, adding cpu: host,hidden=1 to the vm.conf.
I added this line, saved, rebooted, but I still get:

Code:

xxx@Plex:~$ nvidia-smi
No devices were found

Here is my vm.conf:


balloon: 2048
boot: order=scsi0;net0
cores: 8
cpu: host,hidden=1
hostpci0: 0000:2d:00,pcie=1
machine: q35
memory: 8192
name: Plex
net0: virtio=1E:F3:2A:8A:68:A1,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
sata0: /dev/disk/by-id/wwn-0x50014ee602dcbe2e
scsi0: local-lvm:vm-103-disk-0,size=20G
scsihw: virtio-scsi-pci
smbios1: uuid=800ecd99-c378-48eb-8550-c50d823d66d2
sockets: 1
vmgenid: cea01d2b-f6d6-4c9a-9535-35d3ddf19c45

Under root@pve is see the gpu

Code:

01:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P400] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: NVIDIA Corporation GP107GL [Quadro P400]
        Physical Slot: 0
        Flags: bus master, fast devsel, latency 0, IRQ 16
        Memory at fb000000 (32-bit, non-prefetchable) [size=16M]
        Memory at d0000000 (64-bit, prefetchable) [size=256M]
        Memory at e0000000 (64-bit, prefetchable) [size=32M]
        I/O ports at 5000 [size=128]
        Expansion ROM at fc000000 [virtual] [disabled] [size=128K]
        Capabilities: <access denied>
        Kernel driver in use: nvidia
        Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

01:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)
        Subsystem: NVIDIA Corporation GP107GL High Definition Audio Controller
        Physical Slot: 0
        Flags: bus master, fast devsel, latency 0, IRQ 17
        Memory at fc020000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel

and on the Plex VM the NV driver is installed:

Code:

xxx@Plex:~$ modinfo nvidia
filename:       /lib/modules/5.4.0-104-generic/kernel/drivers/video/nvidia.ko
alias:          char-major-195-*
version:        450.80.02
supported:      external
license:        NVIDIA
srcversion:     2132A76E28730AB295AF17B
alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
depends:
retpoline:      Y
name:           nvidia
vermagic:       5.4.0-104-generic SMP mod_unload modversions
parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
parm:           NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp)
parm:           nv_cap_enable_devfs:Enable (1) or disable (0) nv-caps devfs supp                                 ort. Default: 1 (int)
parm:           NVreg_ResmanDebugLevel:int
parm:           NVreg_RmLogonRC:int
parm:           NVreg_ModifyDeviceFiles:int
parm:           NVreg_DeviceFileUID:int
parm:           NVreg_DeviceFileGID:int
parm:           NVreg_DeviceFileMode:int
parm:           NVreg_InitializeSystemMemoryAllocations:int
parm:           NVreg_UsePageAttributeTable:int
parm:           NVreg_MapRegistersEarly:int
parm:           NVreg_RegisterForACPIEvents:int
parm:           NVreg_EnablePCIeGen3:int
parm:           NVreg_EnableMSI:int
parm:           NVreg_TCEBypassMode:int
parm:           NVreg_EnableStreamMemOPs:int
parm:           NVreg_EnableBacklightHandler:int
parm:           NVreg_RestrictProfilingToAdminUsers:int
parm:           NVreg_PreserveVideoMemoryAllocations:int
parm:           NVreg_DynamicPowerManagement:int
parm:           NVreg_DynamicPowerManagementVideoMemoryThreshold:int
parm:           NVreg_EnableUserNUMAManagement:int
parm:           NVreg_MemoryPoolSize:int
parm:           NVreg_KMallocHeapMaxSize:int
parm:           NVreg_VMallocHeapMaxSize:int
parm:           NVreg_IgnoreMMIOCheck:int
parm:           NVreg_NvLinkDisable:int
parm:           NVreg_EnablePCIERelaxedOrderingMode:int
parm:           NVreg_RegisterPCIDriver:int
parm:           NVreg_RegistryDwords:charp
parm:           NVreg_RegistryDwordsPerDevice:charp
parm:           NVreg_RmMsg:charp
parm:           NVreg_GpuBlacklist:charp
parm:           NVreg_TemporaryFilePath:charp
parm:           NVreg_AssignGpus:charp

lshw -c video

Code:

  *-display
       description: VGA compatible controller
       physical id: 1
       bus info: pci@0000:00:01.0
       version: 02
       width: 32 bits
       clock: 33MHz
       capabilities: vga_controller bus_master rom
       configuration: driver=bochs-drm latency=0
       resources: irq:0 memory:f0000000-f0ffffff memory:fe014000-fe014fff memory:c0000-dffff
  *-display
       description: VGA compatible controller
       product: GP107GL [Quadro P400]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:16 memory:fb000000-fbffffff memory:d0000000-dfffffff memory:e0000000-e1ffffff ioport:5000(size=128) memory:fc000000-fc01ffff

During the driver installation I got an error, don't know whether this is the problem or not:

Code:

the CC version check failed
The kernel was built with gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04), but the current
compiler version is cc (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0.

If you have any idea, please help to fix this.

punkyunky · Apr 23, 2022

Try changing the graphics settings in the BIOS of your motherboard (esc on boot). I changed the settings on mine from auto or internal to PCIe and then my VM was able to see the device without any issues. Hope this helps!

leesteken · Apr 23, 2022

Try Primary GPU (xvga=1), which is often necessary for NVidia cards. Try OVMF instead of SeaBIOS
Note that you have enabled ballooning but that cannot work when using passthrough, because all memory must be pinned into actual RAM because of PCI(e) DMA.

Aimsucks · May 30, 2022

leesteken said:
Try Primary GPU (xvga=1), which is often necessary for NVidia cards. Try OVMF instead of SeaBIOS
Note that you have enabled ballooning but that cannot work when using passthrough, because all memory must be pinned into actual RAM because of PCI(e) DMA.

Neither of these worked for me.

Code:

bios: ovmf
boot: order=scsi0;net0
cores: 4
efidisk0: local-lvm:vm-100-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:07:00,pcie=1
machine: q35
memory: 16384
meta: creation-qemu=6.2.0,ctime=1653201783
name: hydra
net0: virtio=5E:F4:43:2C:BE:E4,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: local-lvm:vm-100-disk-0,size=64G
scsi1: dust:vm-100-disk-0,backup=0,discard=on,size=10000G
scsihw: virtio-scsi-pci
smbios1: uuid=d58740ff-401e-4c1c-9b33-e6566a623f9d
sockets: 1
vmgenid: 3e5190c9-ccff-4a10-9b08-78dedc4bd59f

Here is lspci

Code:

$ lspci -nnv -s 01:00
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106GL [Quadro P2000] [10de:1c30] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: NVIDIA Corporation GP106GL [Quadro P2000] [10de:11b3]
        Physical Slot: 0
        Flags: bus master, fast devsel, latency 0, IRQ 16
        Memory at c1000000 (32-bit, non-prefetchable) [size=16M]
        Memory at 800000000 (64-bit, prefetchable) [size=256M]
        Memory at 810000000 (64-bit, prefetchable) [size=32M]
        I/O ports at d000 [size=128]
        Expansion ROM at c2020000 [virtual] [disabled] [size=128K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Legacy Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Capabilities: [250] Latency Tolerance Reporting
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [420] Advanced Error Reporting
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Kernel driver in use: nvidia
        Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

01:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1)
        Subsystem: NVIDIA Corporation GP106 High Definition Audio Controller [10de:11b3]
        Physical Slot: 0
        Flags: bus master, fast devsel, latency 0, IRQ 17
        Memory at c2000000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel

thebrownjedi · Jun 16, 2022

@Aimsucks were you able to figure out the solution? I'm facing the exact error for my Quadro P620.

Aimsucks · Jun 16, 2022

thebrownjedi said:
@Aimsucks were you able to figure out the solution? I'm facing the exact error for my Quadro P620.

https://forum.proxmox.com/threads/g...ems-working-but-it-doesnt.107881/#post-474706

Search

Search

GPU passthrough to Plex VM - nvidia-smi: No devices were found

gyik

Member

punkyunky

New Member

leesteken

Distinguished Member

Aimsucks

New Member

thebrownjedi

New Member

Aimsucks

New Member

GPU passthrough to Plex VM - nvidia-smi: No devices were found

gyik

Member

​

punkyunky

New Member

leesteken

Distinguished Member

Aimsucks

New Member

thebrownjedi

New Member

Aimsucks

New Member