Promxox4.4: QM Status - Stopped for Passthrough of IBM1015

[schmyd]

Hi Guys,

I am trying to passthrough my IBM1015 to Freenas10, crossflashed to 9211IT-mode without any ROM.

However, I am unable to start the VM and don't have any useful error messages:

root@pve:~# qm start 100
root@pve:~# qm status 100
status: stopped

Syslog gives me:

Feb 10 20:44:12 pve systemd[1]: Starting 100.scope.
Feb 10 20:44:12 pve systemd[1]: Started 100.scope.
Feb 10 20:44:13 pve kernel: [ 1514.197278] device tap100i0 entered promiscuous mode
Feb 10 20:44:13 pve kernel: [ 1514.200786] vmbr0: port 2(tap100i0) entered forwarding state
Feb 10 20:44:13 pve kernel: [ 1514.200793] vmbr0: port 2(tap100i0) entered forwarding state
Feb 10 20:44:14 pve kernel: [ 1515.579343] vfio-pci 0000:04:00.0: enabling device (0400 -> 0403)
Feb 10 20:44:14 pve qm[4436]: <root@pam> end task UPID:pve:00001155:00024F49:589E180C:qmstart:100:root@pam: OK
Feb 10 20:44:15 pve kernel: [ 1516.394459] vmbr0: port 2(tap100i0) entered disabled state
Feb 10 20:44:15 pve kernel: [ 1516.394512] vmbr0: port 2(tap100i0) entered disabled state

The VM is configured as

balloon: 0
bootdisk: ide0
cores: 2
cpu: host
hostpci0: 04:00.0,pcie=1
ide0: local-zfs:vm-100-disk-1,cache=writethrough,size=10G
ide2: local:iso/FreeNAS-10-MASTER-201702030500-fcfa5e9c5.iso,media=cdrom
machine: q35
memory: 8192
name: FN10
net0: e1000=72:1C:51:AD:96:2A,bridge=vmbr0
numa: 1
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=92e61ac4-c669-4385-bed1-7d8fb2ae2723
sockets: 1

My hardware is

root@pve:~# lspci | grep LSI
04:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 02)

In grub I have

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

I could see all the attached drives in proxmox until I blocked the driver via

blacklist mpt3sas

IOMMU also looks good - or does it?

root@pve:~# dmesg | grep -e DMAR -e IOMMU
[    0.000000] ACPI: DMAR 0x00000000D87FE0F0 0000B8 (v01 INTEL  HSW      00000001 INTL 00000001)
[    0.000000] DMAR: Disable GFX device mapping
[    0.000000] DMAR: IOMMU enabled
[    0.027439] DMAR: Host address width 39
[    0.027440] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.027446] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.027447] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.027449] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da
[    0.027450] DMAR: RMRR base: 0x000000dbb7b000 end: 0x000000dbb88fff
[    0.027451] DMAR: RMRR base: 0x000000dd000000 end: 0x000000df1fffff
[    0.027452] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1
[    0.027453] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.027454] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.027611] DMAR: DRHD: handling fault status reg 2
[    0.027635] DMAR: DMAR:[DMA Read] Request device [00:1f.2] fault addr dbb41000
DMAR:[fault reason 06] PTE Read access is not set
[    0.027737] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.480173] DMAR: No ATSR found
[    0.480228] DMAR: dmar1: Using Queued invalidation
[    0.480237] DMAR: Setting RMRR:
[    0.480250] DMAR: Setting identity map for device 0000:00:14.0 [0xdbb7b000 - 0xdbb88fff]
[    0.480268] DMAR: Setting identity map for device 0000:00:1a.0 [0xdbb7b000 - 0xdbb88fff]
[    0.480284] DMAR: Setting identity map for device 0000:00:1d.0 [0xdbb7b000 - 0xdbb88fff]
[    0.480296] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.480302] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.480381] DMAR: Intel(R) Virtualization Technology for Directed I/O

Any advice on the issue? It's hard for me to find an entry point for further debugging.

Thanks in Advance!

 

[schmyd]

Update: The Problem seems to be a failed DMA Mapping

kvm: VFIO_MAP_DMA: -14
kvm: vfio_dma_map(0x7fbe7c22c500, 0xfdf20000, 0x2000, 0x7fbeaedd4000) = -14 (Bad address)
qemu: hardware error: vfio: DMA mapping failed, unable to continue
CPU #0:
EAX=00000103 EBX=80068004 ECX=00000103 EDX=00000cfc
ESI=00068000 EDI=00000000 EBP=1efabd90 ESP=00006d74
EIP=1efad71a EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000f71a0 00000037
IDT=     000f71de 00000000
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000

Any idea on this?

 

[naglerrr]

Is there any news on this matter? I have exactly the same issue. The problem also occurs when using Proxmox 5.0 beta.

 

[quangnhut123]

Mee to for passthrough NIC on pve 4.4

 

[naglerrr]

I bought a LSI 9207-8i because passing through my crossflashed M1015 would not work. Passing the 9207-8i worked right away on first attempt. I has now been running for two days without an issue.

 

[mueslo]

I bought a LSI 9207-8i because passing through my crossflashed M1015 would not work. Passing the 9207-8i worked right away on first attempt. I has now been running for two days without an issue.

Just grabbed a cheap used Dell OEM 9217-8i, flashed it to IT mode, works right away! Finally able to upgrade from 3.4 to 5.x