[SOLVED] GPU passthrough not working - Ubuntu 22.04, RTX 3090, Proxmox 8.1.2, H12ssl-i + Epyc 7642

[lfrankqy]

Hello, I'm new to Proxmox. I'm trying to run a Ubuntu 22.04 VM under proxmox8.1.2 with RTX 3090 GPU passthrough to do some experiments in AI. But Ollama is not able to find the GPU. Below is the output of the Ollama log.

Aug 25 04:36:02 aiserver ollama[42354]: time=2024-08-25T04:36:02.121Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]"
Aug 25 04:36:02 aiserver ollama[42354]: time=2024-08-25T04:36:02.121Z level=INFO source=gpu.go:204 msg="looking for compatible GPUs"
Aug 25 04:36:02 aiserver ollama[42354]: time=2024-08-25T04:36:02.142Z level=INFO source=gpu.go:560 msg="no nvidia devices detected" library=/usr/lib/x86_64-linux-gnu/libcuda.so.560.35.03
Aug 25 04:36:02 aiserver ollama[42354]: time=2024-08-25T04:36:02.161Z level=INFO source=gpu.go:350 msg="no compatible GPUs were discovered"
Aug 25 04:36:02 aiserver ollama[42354]: time=2024-08-25T04:36:02.161Z level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="38.2 GiB" available="37.4 GiB"

Below is the output of lspci, which shows the GPU is present:

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3090] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GA102 High Definition Audio Controller (rev a1)
05:01.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge

The dkms status shows the driver has been loaded.

nvidia/560.35.03, 5.15.0-119-generic, x86_64: installed

But when I run nvidia-smi, it complains about could not communicate with the driver.

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Attached is my VM config file:

agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 16
cpu: host,hidden=1,flags=+pcid
efidisk0: vmdisk01:vm-102-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:81:00,pcie=1
ide2: local:iso/ubuntu-22.04.4-live-server-amd64.iso,media=cdrom,size=2055086K
machine: q35
memory: 40000
meta: creation-qemu=8.1.5,ctime=1724552572
name: aiserver
net0: virtio=BC:24:11:9D:0D:28,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: vmdisk01:vm-102-disk-1,iothread=1,size=500G
scsihw: virtio-scsi-single
smbios1: uuid=aace19ba-f541-48e1-bdea-ce92b59e3720
sockets: 1
vga: std
vmgenid: 4c74fab0-81cc-47a8-aab3-8bd2cbb54b88

Could someone help me with this issue?

Thanks in advance,
Frank

 

[lfrankqy]

FYI - the issue has been solved. I should disable secure boot.

Below is the process of how I make it work. Just in case someone needs it:

https://truthful-era-6d5.notion.sit...roxmox-9161462ec1ef4c9ab302a322adb5c5bb?pvs=4

 

[kbirger]

How do you get the VM gui to modify bios settings? And is there a way to do it headlessly?

 

[lespaul]

How do you get the VM gui to modify bios settings? And is there a way to do it headlessly?

The simplest way is to use the PVE web GUI, in your VM > Console, reboot the machine and watch for the proxmox boot logo.

Press escape.

 

[kbirger]

That would be the host BIOS, right? I thought we were talking about the container BIOS