[SOLVED] all iommugroup are -1

E

ecotechie

Member
Nov 11, 2023
36
1
8
[SOLVED] Gotta run update-grub for changes to take effect :rolleyes:

Hi, I am not able to figure out why I can't do a PCI pass through of the two Western Digital SSDs to a VM (or LXC for that matter).

My motherboard is a Gigabyte Z790 AORUS ELITE AX

I tried to add them to a VM but get this error... I did enable VT-d on the BIOS.

1704332944764.png

Code: 
root@pod:~# dmesg | grep -e DMAR -e IOMMU
[    0.016408] ACPI: DMAR 0x00000000342BA000 000088 (v01 INTEL  EDK2     00000002      01000013)
[    0.016458] ACPI: Reserving DMAR table memory at [mem 0x342ba000-0x342ba087]
[    0.196574] DMAR: Host address width 39
[    0.196575] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.196579] DMAR: dmar0: reg_base_addr fed90000 ver 4:0 cap 1c0000c40660462 ecap 29a00f0505e
[    0.196581] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.196586] DMAR: dmar1: reg_base_addr fed91000 ver 5:0 cap d2008c40660462 ecap f050da
[    0.196588] DMAR: RMRR base: 0x0000003c000000 end: 0x000000403fffff
[    0.196590] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.196591] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.196592] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.197130] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    2.037540] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
[    6.160160] AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug.

Code: 
root@pod:~# dmesg | grep 'remapping'
[    0.196592] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.197130] DMAR-IR: Enabled IRQ remapping in x2apic mode

Code: 
root@pod:~# cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
#GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX=""

Code: 
root@pod:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.
vfio
vfio_iommu_type1
vfio_pci

Code: 
root@pod:~# pvesh get /nodes/pod/hardware/pci --pci-class-blacklist ""
┌──────────┬────────┬───────────────┬────────────┬────────┬─────────────────────────────────────────────────────────────────┬──────┬──────────────────┬───────────────────────┬──────────────────┬────────────────────────────────────────┬────────────────────────────────────────┐
│ class    │ device │ id            │ iommugroup │ vendor │ device_name                                                     │ mdev │ subsystem_device │ subsystem_device_name │ subsystem_vendor │ subsystem_vendor_name                  │ vendor_name                            │
╞══════════╪════════╪═══════════════╪════════════╪════════╪═════════════════════════════════════════════════════════════════╪══════╪══════════════════╪═══════════════════════╪══════════════════╪════════════════════════════════════════╪════════════════════════════════════════╡
│ 0x010400 │ 0xa77f │ 0000:00:0e.0  │         -1 │ 0x8086 │ Volume Management Device NVMe RAID Controller Intel Corporation │      │ 0x0000           │                       │ 0x8086           │ Intel Corporation                      │ Intel Corporation                      │
├──────────┼────────┼───────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x010601 │ 0x7a62 │ 10000:e0:17.0 │         -1 │ 0x8086 │                                                                 │      │ 0xb005           │                       │ 0x1458           │ Gigabyte Technology Co., Ltd           │ Intel Corporation                      │
├──────────┼────────┼───────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x010802 │ 0x5416 │ 10000:e1:00.0 │         -1 │ 0x1344 │                                                                 │      │ 0x1100           │                       │ 0x1344           │ Micron Technology Inc                  │ Micron Technology Inc                  │
├──────────┼────────┼───────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x010802 │ 0x5030 │ 10000:e2:00.0 │         -1 │ 0x15b7 │ Western Digital WD Black SN850X NVMe SSD                        │      │ 0x5030           │                       │ 0x15b7           │ Sandisk Corp                           │ Sandisk Corp                           │
├──────────┼────────┼───────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x010802 │ 0x5030 │ 10000:e3:00.0 │         -1 │ 0x15b7 │ Western Digital WD Black SN850X NVMe SSD                        │      │ 0x5030           │                       │ 0x15b7           │ Sandisk Corp                           │ Sandisk Corp                           │
├──────────┼────────┼───────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x020000 │ 0x8125 │ 0000:06:00.0  │         -1 │ 0x10ec │ RTL8125 2.5GbE Controller                                       │      │ 0xe000           │                       │ 0x1458           │ Gigabyte Technology Co., Ltd           │ Realtek Semiconductor Co., Ltd.        │
├──────────┼────────┼───────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x028000 │ 0x7a70 │ 0000:00:14.3  │         -1 │ 0x8086 │
 
Last edited:
looks like I mess-up with something


I was able to see my hard disks on the node disks,,,now I only see the nvme which boots the system...please help
 
Screenshot 2024-03-23 at 7.33.37 PM.pngwas able to restore it (was following the guid of gpu pass thru looks like I mess-up something ) but still can not see making ...only raw
 
Code: 
root@prox:~# pvesh get /nodes/prox/hardware/pci --pci-class-blacklist ""
┌──────────┬────────┬──────────────┬────────────┬────────┬────────────────────────────────────────────────────┬──────┬──────────────────┬────
│ class    │ device │ id           │ iommugroup │ vendor │ device_name                                        │ mdev │ subsystem_device │ sub
╞══════════╪════════╪══════════════╪════════════╪════════╪════════════════════════════════════════════════════╪══════╪══════════════════╪════
│ 0x010601 │ 0xa352 │ 0000:00:17.0 │         -1 │ 0x8086 │ Cannon Lake PCH SATA AHCI Controller               │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x010802 │ 0xa80a │ 0000:01:00.0 │         -1 │ 0x144d │ NVMe SSD Controller PM9A1/PM9A3/980PRO             │      │ 0xa801           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x020000 │ 0x15bb │ 0000:00:1f.6 │         -1 │ 0x8086 │ Ethernet Connection (7) I219-LM                    │      │ 0x83e0           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x030000 │ 0x3e92 │ 0000:00:02.0 │         -1 │ 0x8086 │ CoffeeLake-S GT2 [UHD Graphics 630]                │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x040300 │ 0xa348 │ 0000:00:1f.3 │         -1 │ 0x8086 │ Cannon Lake PCH cAVS                               │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x050000 │ 0xa36f │ 0000:00:14.2 │         -1 │ 0x8086 │ Cannon Lake PCH Shared SRAM                        │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x060000 │ 0x3ec2 │ 0000:00:00.0 │         -1 │ 0x8086 │ 8th Gen Core Processor Host Bridge/DRAM Registers  │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x060100 │ 0xa306 │ 0000:00:1f.0 │         -1 │ 0x8086 │ Q370 Chipset LPC/eSPI Controller                   │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x060400 │ 0xa32c │ 0000:00:1b.0 │         -1 │ 0x8086 │ Cannon Lake PCH PCI Express Root Port #21          │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x070002 │ 0xa363 │ 0000:00:16.3 │         -1 │ 0x8086 │ Cannon Lake PCH Active Management Technology - SOL │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x078000 │ 0xa360 │ 0000:00:16.0 │         -1 │ 0x8086 │ Cannon Lake PCH HECI Controller                    │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x0c0330 │ 0xa36d │ 0000:00:14.0 │         -1 │ 0x8086 │ Cannon Lake PCH USB 3.1 xHCI Host Controller       │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x0c0500 │ 0xa323 │ 0000:00:1f.4 │         -1 │ 0x8086 │ Cannon Lake PCH SMBus Controller                   │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x0c8000 │ 0xa324 │ 0000:00:1f.5 │         -1 │ 0x8086 │ Cannon Lake PCH SPI Controller                     │      │ 0x83e1           │   
├──────────┼────────┼──────────────┼────────────┼────────┼────────────────────────────────────────────────────┼──────┼──────────────────┼────
│ 0x118000 │ 0xa379 │ 0000:00:12.0 │         -1 │ 0x8086 │ Cannon Lake PCH Thermal Controller                 │      │ 0x83e1           │   
└──────────┴────────┴──────────────┴────────────┴────────┴────────────────────────────────────────────────────┴──────┴──────────────────┴────
root@prox:~#
this is my output...what am I missing?
 
nano /etc/modprobe.d/vfio.conf

I just added the options of the vendor ID

options vfio-pci ids=8086:a352

According to some guide over reddit
 
looks like the above is the problem
Code: 
options vfio-pci ids=8086:a352

however removing it still have the same problem with no mapping
 
arlsharawi said:
BOOT_IMAGE=/boot/vmlinuz-6.5.13-3-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt
Click to expand...
In that case, VT-d is not enabled in the motherboard BIOS, or the motherboard does not support VT-d, or the CPU does not support VT-d. Check the Intel product specifications and/or contact your motherboard manufacturer for support.
 
now Im not able to see the HDD even with removing the options ID looks like I used the blacklist command wrong..how to troubleshoot?

as it shows at the boot but then HDD disappears
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom