Proxmox build passtrough issues

[JORGITO_YKE]

Hi, before writing this lines, i've spent 2 enterily days trying to make work this hardware and reading a lot. So now, i think i need some experienced help.

ENVIRONMENT

Proxmox 5.4 because latest one doesn't work anything related to vfio driver assign to gpu.

HARDWARE

I'm using two computers in one but the proxmox host contains the following:

- Gigabyte x99 ud5 wifi
- Nvidia gtx1080ti
- Radeon r9 390x
- Intel e5-2690 v3
- A bunch of disks, but the most important is that i have a disk for proxmox and another two for each vm.

DOUBTS/QUESTIONS

I need another gpu? I'am going to pass each card above listed to two diferent virtual machines. If true, I will buy it.
Card is Matrox G550 pci x1 with two DVI.

I will need a ghost screen adapter?

GUIDE

The best tutorial that aproaches to my goal is this one, so I've followed it but with no success.

https://forum.proxmox.com/threads/proxmox-pcie-passthrough-my-experience.30505/

PROBLEMS

The problem is when I pass the GPU to the virtual machine, crashes and the host too. Only tested by now with Radeon because Nvidia is my first GPU, but it should work the same way.

CONFIGS/OUTPUTS

These are my configs:

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

/etc/modprobe.d/blacklist.conf

blacklist radeon
blacklist amdgpu

/etc/modprobe.d/vfio.conf

options vfio-pci ids=1002:67b0,1002:aac8 disable=vga=1

These are my outputs:

lspci -nnv | grep 390X
02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] [1002:67b0] (rev 80) (prog-if 00 [VGA controller])
        Subsystem: PC Partner Limited / Sapphire Technology Grenada XT2 [Radeon R9 390X] [174b:e324]
02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]
        Subsystem: PC Partner Limited / Sapphire Technology Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [174b:aac8]

[CODE=bash]lspci -v
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] (rev 80) (prog-if 00 [VGA controller])
        Subsystem: PC Partner Limited / Sapphire Technology Grenada XT2 [Radeon R9 390X]
        Physical Slot: 6
        Flags: fast devsel, IRQ 11, NUMA node 0
        Memory at e0000000 (64-bit, prefetchable) [disabled] [size=256M]
        Memory at f0000000 (64-bit, prefetchable) [disabled] [size=8M]
        I/O ports at d000 [disabled] [size=256]
        Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=256K]
        Expansion ROM at fb840000 [disabled] [size=128K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [200] #15
        Capabilities: [270] #19
        Capabilities: [2b0] Address Translation Service (ATS)
        Capabilities: [2c0] Page Request Interface (PRI)
        Capabilities: [2d0] Process Address Space ID (PASID)
        Kernel driver in use: vfio-pci
        Kernel modules: radeon, amdgpu

02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X]
        Subsystem: PC Partner Limited / Sapphire Technology Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X]
        Physical Slot: 6
        Flags: bus master, fast devsel, latency 0, IRQ 10, NUMA node 0
        Memory at fb860000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

If you need anything that is not listed here, please tell me and I will post whatever you need. Thanks in advance.

 

[dcsapak]

The problem is when I pass the GPU to the virtual machine, crashes and the host too.

how do the iommu groups look like? i guess what happens is that both gpus are in the same group and this will not work...

also:

Proxmox 5.4 because latest one doesn't work anything related to vfio driver assign to gpu.

what exactly is the problem with 6.x ?

 

[JORGITO_YKE]

Sorry for answering so late. I've ended using esxi, it worked for my needings.

what exactly is the problem with 6.x ?

The problem was at the time of adding gpu to vfio.conf.

It is supposed when they are blacklisted and at the moment of restart, vfio driver should be present. Not in my case. Always was the radeon/amdgpu module enabled when you do lspci -v in version 6.x.

In version 5.4 works as expected. Blacklist, then vfio.conf, then lspci -v and output was Kernel driver in use: vfio-pci as expected.

However, thank you for the answer, I would take a look in deep for another system that I am going to build.

Thank you so much.