PCI passthrough for SATA controller

S

Spike42

Member
Nov 27, 2009
3
0
21
Hi,
I'm trying to passthrough an ASRock SATA3 PCIe controller to a Win7 VM.

My hardware:
Intel Core i5-2500
16GB RAM
MSI H67MA-E45
ASRock SATA3 Card

What I did so far:
- enabled vt-d in BIOS
- enabled PCI passthrough in GRUB as shown in the wiki
- added "hostpci0: 02:00.0" and "hostpci1: 02:00.1" in "/etc/pve/qemu-server/100.conf"

Result: The VM doesn't start anymore.

Can anyone tell my what to do/try next, please?


Additional Info:
pveversion -v
Code: 
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-12-pve
proxmox-ve-2.6.32: 2.1-68
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-12-pve: 2.6.32-68
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-16
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1

lspci
Code: 
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 3 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation H67 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9120 SATA 6Gb/s Controller (rev 12)
02:00.1 IDE interface: Marvell Technology Group Ltd. 88SE91A4 SATA 6Gb/s Controller (rev 12)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
04:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)

lspci -vv
Code: 
02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9120 SATA 6Gb/s Controller (rev 12) (prog-if 01 [AHCI 1.0])
        Subsystem: Marvell Technology Group Ltd. 88SE9120 SATA 6Gb/s Controller
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 18
        Region 0: I/O ports at e090 [size=8]
        Region 1: I/O ports at e080 [size=4]
        Region 2: I/O ports at e070 [size=8]
        Region 3: I/O ports at e060 [size=4]
        Region 4: I/O ports at e050 [size=16]
        Region 5: Memory at fe521000 (32-bit, non-prefetchable) [size=2K]
        Expansion ROM at fe510000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: fee00398  Data: 0000
        Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <64us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Kernel driver in use: pci-stub

02:00.1 IDE interface: Marvell Technology Group Ltd. 88SE91A4 SATA 6Gb/s Controller (rev 12) (prog-if 8f [Master SecP SecO PriP PriO])
        Subsystem: Marvell Technology Group Ltd. 88SE91A4 SATA 6Gb/s Controller
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin B routed to IRQ 19
        Region 0: I/O ports at e040 [disabled] [size=8]
        Region 1: I/O ports at e030 [disabled] [size=4]
        Region 2: I/O ports at e020 [disabled] [size=8]
        Region 3: I/O ports at e010 [disabled] [size=4]
        Region 4: I/O ports at e000 [disabled] [size=16]
        Region 5: Memory at fe520000 (32-bit, non-prefetchable) [disabled] [size=16]
        Expansion ROM at fe500000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: fee003f8  Data: 0000
        Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <64us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Kernel driver in use: pci-stub

lspci -n (02:00.0 and 02:00.1 should be the ASRock controller)
Code: 
00:00.0 0600: 8086:0100 (rev 09)
00:02.0 0300: 8086:0102 (rev 09)
00:16.0 0780: 8086:1c3a (rev 04)
00:1a.0 0c03: 8086:1c2d (rev 05)
00:1b.0 0403: 8086:1c20 (rev 05)
00:1c.0 0604: 8086:1c10 (rev b5)
00:1c.2 0604: 8086:1c14 (rev b5)
00:1c.4 0604: 8086:1c18 (rev b5)
00:1c.6 0604: 8086:1c1c (rev b5)
00:1d.0 0c03: 8086:1c26 (rev 05)
00:1f.0 0601: 8086:1c4a (rev 05)
00:1f.2 0106: 8086:1c02 (rev 05)
00:1f.3 0c05: 8086:1c22 (rev 05)
02:00.0 0106: 1b4b:9120 (rev 12)
02:00.1 0101: 1b4b:91a4 (rev 12)
03:00.0 0200: 10ec:8168 (rev 06)
04:00.0 0c03: 1033:0194 (rev 04)

dmesg | grep -i iommu
Code: 
Command line: BOOT_IMAGE=/vmlinuz-2.6.32-12-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.32-12-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
Intel-IOMMU: enabled
IOMMU fed90000: ver 1:0 cap c0000020e60262 ecap f0101a
IOMMU fed91000: ver 1:0 cap c9008020660262 ecap f0105a
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node 0
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
IOMMU 0xfed90000: using Queued invalidation
IOMMU 0xfed91000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:02.0 [0xbb800000 - 0xbfa00000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbaddc000 - 0xbadef000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbaddc000 - 0xbadef000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1
alloc irq_2_iommu on node -1

dmesg after trying to start the vm
Code: 
pci-stub 0000:02:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100003)
pci-stub 0000:02:00.1: restoring config space at offset 0x1 (was 0x100400, writing 0x100000)
device tap100i1 entered promiscuous mode
vmbr0: port 2(tap100i1) entering forwarding state
New device tap100i1 does not support netpoll
Disabling netpoll for vmbr0
pci-stub 0000:02:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
pci-stub 0000:02:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100003)
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 32 for MSI/MSI-X
pci-stub 0000:02:00.1: enabling device (0000 -> 0003)
pci-stub 0000:02:00.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
pci-stub 0000:02:00.1: restoring config space at offset 0x1 (was 0x100400, writing 0x100003)
assign device: host bdf = 2:0:1
pci-stub 0000:02:00.1: irq 40 for MSI/MSI-X
pci-stub 0000:02:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100403)
pci-stub 0000:02:00.1: restoring config space at offset 0x1 (was 0x100400, writing 0x100403)
vmbr0: port 2(tap100i1) entering disabled state
vmbr0: port 2(tap100i1) entering disabled state
pci-stub 0000:02:00.1: restoring config space at offset 0x1 (was 0x100400, writing 0x100000)
pci-stub 0000:02:00.1: PCI INT B disabled
pci-stub 0000:02:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100003)
pci-stub 0000:02:00.0: PCI INT A disabled
 
Last edited:
Thanks for your reply, Dietmar!

Using "hostpci0" two times was a copy and paste error.
In the configuration file I used hostpci0 and hostpci1. I corrected my post above.

My first try was to passthrough only "hostpci0: 02:00.0" - that didn't worked either.
But I think I have to passthrough both, 02:00.0 and 02:00.1, right? (see entries for Marvell SATA 6Gb/s Controller in lspci output).

Do you have any other hint what I can try?
Is the dmesg output any help?
 
You must log in or register to reply here.
Top Bottom