Im having some issues passing through a LSI 9300-8i HBA to Nappit running under OmniOS.
The cards do not have bootable ROMs installed. I followed the how-to. The 2 connected SSDs are visible in Nappit VM but when trying to create a pool I see the following errors before Napp-it reports an error.
Heres the section from the lspci -vnn
tried config with both 'machine: q35' and without.
I see this illegal PCI request error on booting the VM
And finally heres the lspci output from the proxmox VM
The cards do not have bootable ROMs installed. I followed the how-to. The 2 connected SSDs are visible in Nappit VM but when trying to create a pool I see the following errors before Napp-it reports an error.
Code:
Dec 9 19:48:46 nappit scsi; WARNING: /pci@0.0/pci1000,30e0@10 (mph_sas0):
Dec 9 19:48:46 nappit MPT Firmware Fault, code: 2622Heres the section from the lspci -vnn
Code:
01:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)
Subsystem: LSI Logic / Symbios Logic SAS9300-8i [1000:30e0]
Flags: bus master, fast devsel, latency 0, IRQ 36
I/O ports at e000 [disabled] [size=256]
Memory at fb840000 (64-bit, non-prefetchable) [size=64K]
Memory at fb800000 (64-bit, non-prefetchable) [size=256K]
Expansion ROM at fb700000 [size=1M]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [a8] MSI: Enable+ Count=1/1 Maskable+ 64bit+
Capabilities: [c0] MSI-X: Enable- Count=96 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [1e0] #19
Capabilities: [1c0] Power Budgeting <?>
Capabilities: [190] #16
Capabilities: [148] Alternative Routing-ID Interpretation (ARI)
Kernel driver in use: vfio-pci
02:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)
Subsystem: LSI Logic / Symbios Logic SAS9300-8i [1000:30e0]
Flags: fast devsel, IRQ 36
I/O ports at d000 [disabled] [size=256]
Memory at fb640000 (64-bit, non-prefetchable) [disabled] [size=64K]
Memory at fb600000 (64-bit, non-prefetchable) [disabled] [size=256K]
Expansion ROM at fb500000 [disabled by cmd] [size=1M]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [a8] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [c0] MSI-X: Enable- Count=96 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [1e0] #19
Capabilities: [1c0] Power Budgeting <?>
Capabilities: [190] #16
Capabilities: [148] Alternative Routing-ID Interpretation (ARI)
Kernel driver in use: vfio-pci
Code:
root@proxmox:~# dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x000000007E1C6F70 0000CC (v01 A M I OEMDMAR 00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.061930] DMAR: Host address width 46
[ 0.061931] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[ 0.061935] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0466 ecap f020de
[ 0.061936] DMAR: RMRR base: 0x0000007f235000 end: 0x0000007f243fff
[ 0.061937] DMAR: ATSR flags: 0x0
[ 0.061938] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[ 0.061939] DMAR-IR: IOAPIC id 0 under DRHD base 0xfbffc000 IOMMU 0
[ 0.061940] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0
[ 0.061941] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[ 0.061942] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.062126] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.785142] DMAR: dmar0: Using Queued invalidation
[ 0.785155] DMAR: Setting RMRR:
[ 0.785172] DMAR: Setting identity map for device 0000:00:1a.0 [0x7f235000 - 0x7f243fff]
[ 0.785193] DMAR: Setting identity map for device 0000:00:1d.0 [0x7f235000 - 0x7f243fff]
[ 0.785205] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.785213] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.785223] DMAR: Intel(R) Virtualization Technology for Directed I/Otried config with both 'machine: q35' and without.
Code:
balloon: 0
bootdisk: ide0
cores: 2
cpu: host
ide0: local-zfs:vm-114-disk-1,size=32G
ide2: none,media=cdrom
memory: 16384
name: napp
net0: virtio=DE:DC:DC:5E:6E:B3,bridge=vmbr20
numa: 0
ostype: solaris
scsihw: virtio-scsi-pci
smbios1: uuid=c34bd90a-5c09-4ff5-8629-81ba7b8c3194
sockets: 1
hostpci0: 01:00.0
hostpci1: 02:00.0I see this illegal PCI request error on booting the VM
And finally heres the lspci output from the proxmox VM
Code:
Last login: Sat Dec 10 12:45:16 2016 on console
OmniOS 5.11 omnios-r151020-b5b8c75 November 2016
root@napp:/root# lspci -vnn
00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237] (rev 02)
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Flags: fast devsel
00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000]
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Flags: bus master, medium devsel, latency 0
00:01.1 IDE interface [0101]: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] [8086:7010] (prog-if 80 [Master])
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Flags: bus master, medium devsel, latency 0
I/O ports at e240
00:01.2 USB controller [0c03]: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] [8086:7020] (rev 01) (prog-if 00 [UHCI])
Subsystem: Red Hat, Inc QEMU Virtual Machine [1af4:1100]
Flags: bus master, fast devsel, latency 0, IRQ 11
I/O ports at e200
00:01.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 03)
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Flags: medium devsel, IRQ 9
00:02.0 VGA compatible controller [0300]: Cirrus Logic GD 5446 [1013:00b8] (prog-if 00 [VGA controller])
Subsystem: Red Hat, Inc QEMU Virtual Machine [1af4:1100]
Flags: bus master, fast devsel, latency 0
Memory at fa000000 (32-bit, prefetchable)
Memory at feaf0000 (32-bit, non-prefetchable)
Expansion ROM at feac0000 [disabled]
00:10.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)
Subsystem: LSI Logic / Symbios Logic SAS9300-8i [1000:30e0]
Flags: bus master, fast devsel, latency 0, IRQ 11
I/O ports at e000 [disabled]
Memory at fead0000 (64-bit, non-prefetchable)
Memory at fea00000 (64-bit, non-prefetchable)
Expansion ROM at fe800000 [disabled]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [a8] MSI: Enable+ Count=1/1 Maskable+ 64bit+
Capabilities: [c0] MSI-X: Enable- Count=96 Masked-
00:11.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)
Subsystem: LSI Logic / Symbios Logic SAS9300-8i [1000:30e0]
Flags: bus master, fast devsel, latency 0, IRQ 10
I/O ports at e100 [disabled]
Memory at feae0000 (64-bit, non-prefetchable)
Memory at fea40000 (64-bit, non-prefetchable)
Expansion ROM at fe900000 [disabled]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [a8] MSI: Enable+ Count=1/1 Maskable+ 64bit+
Capabilities: [c0] MSI-X: Enable- Count=96 Masked-
00:12.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device [1af4:1000]
Subsystem: Red Hat, Inc Device [1af4:0001]
Flags: bus master, fast devsel, latency 0, IRQ 10
I/O ports at e220
Memory at feaf1000 (32-bit, non-prefetchable)
Memory at fc000000 (64-bit, prefetchable)
Expansion ROM at fea80000 [disabled]
Capabilities: [98] MSI-X: Enable+ Count=3 Masked-
Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
Capabilities: [70] Vendor Specific Information: VirtIO: Notify
Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
Capabilities: [50] Vendor Specific Information: VirtIO: ISR
Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
00:1e.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCI-PCI bridge [1b36:0001] (prog-if 00 [Normal decode])
Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 10
Memory at feaf2000 (64-bit, non-prefetchable)
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000d000-0000dfff
Memory behind bridge: fe600000-fe7fffff
Prefetchable memory behind bridge: 00000000fca00000-00000000fcbfffff
Capabilities: [4c] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [48] Slot ID: 0 slots, First+, chassis 01
Capabilities: [40] Hot-plug capable
00:1f.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCI-PCI bridge [1b36:0001] (prog-if 00 [Normal decode])
Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 11
Memory at feaf3000 (64-bit, non-prefetchable)
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fe400000-fe5fffff
Prefetchable memory behind bridge: 00000000fc800000-00000000fc9fffff
Capabilities: [4c] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [48] Slot ID: 0 slots, First+, chassis 02
Capabilities: [40] Hot-plug capable
Last edited: