PCI Passthrough

A

adamb

Famous Member
Proxmox Subscriber
Mar 1, 2012
1,323
73
113
Hey all I am trying to pass through a couple 4 port HBA cards to my VM. I can't seem to get this working with the VM config file, but I can add them in after it boots with qemu. Below is my VM config file.

bootdisk: virtio0
cores: 4
cpu: host
memory: 25000
name: monthvault
net0: virtio=22:A6:54:77:92:A8,bridge=vmbr1
ostype: l26
sockets: 1
virtio0: drbd1:vm-101-disk-1,size=100G
hostpci0: 86:00.0
hostpci1: 8b:00.0

If I use the following command after the VM boots, then all is well.

device_add pci-assign,host=86:00.0,id=hba1

Am I missing something in the config file, or has something changed that the wiki hasn't been updated with? I appreciate the help!
 
No one uses pci pass through? This seems very straight forward it should work. Like I said I can manually add the pci device once the VM is started and everything works great. When I add the lines to the vm config though it fails to boot.
 
I notice I am getting the following error apon boot.

Unable to load LSI Corporation MPT BIOS
MPT BIOS Fault 02h encountered at adapter PCI(00h,04h,00h)
Firmware Fault Code: 2667h

I wonder if it would be worth updateing the firmware on the card itself. Appreciate any input on this one.
 
I updated both cards to the latest firmware from LSI. Still same issue.

Just a recap
VM gets "Unable to load LSI corporation MPT BIOS" message when booting
If I boot without the hostpci entries in the VM config all is well
I can then manually add the devices though the monitor tab
If I reboot after manually adding them, I run into the same issue as when they are located in the VM config file.
 
hi adamb,

I aktually don`t use this setup - but plan to do so. As I read your message I can imagine that is a Problem for the seabios of KVM-machines to support a RAID-controller completely. But this is not the fault of seabios, because it probably never was created to support RAID-controller.

Especially for use with ZFS-Storages it is recommended to flash the so called "IT-Mode"-firmware to the LSI Controller. This makes the controller to a quite dump device without BIOS-specific extensions. You are not able to boot from devices connected to this controller etc.

For my storage boxes I adapted the following links and flashed two LSI 9211-9i into the IT-mode.

http://www.servethehome.com/ibm-serveraid-m1015-part-4/
or
http://blog.grem.de/sysadmin/LSI-SAS2008-Flashing-2012-04-12-22-17.html

Hope it leads to the correct direction
 
vmanz said:
hi adamb,

I aktually don`t use this setup - but plan to do so. As I read your message I can imagine that is a Problem for the seabios of KVM-machines to support a RAID-controller completely. But this is not the fault of seabios, because it probably never was created to support RAID-controller.

Especially for use with ZFS-Storages it is recommended to flash the so called "IT-Mode"-firmware to the LSI Controller. This makes the controller to a quite dump device without BIOS-specific extensions. You are not able to boot from devices connected to this controller etc.

For my storage boxes I adapted the following links and flashed two LSI 9211-9i into the IT-mode.

http://www.servethehome.com/ibm-serveraid-m1015-part-4/
or
http://blog.grem.de/sysadmin/LSI-SAS2008-Flashing-2012-04-12-22-17.html

Hope it leads to the correct direction
Click to expand...

I am utilizing a SAS 9201-16e HBA that doesn't have any raid support. It comes with the IT firmware as is. I just flashed it to the latest IT firmware and that seemed to help a little, I also disabled the boot options within the LSI configuration. I still can't add the pci cards from within the vm config file. However I can add them through the monitor tab and now it will survive a reboot. There are some new and interesting errors in the seabios when booting.

MPT BIOS Fault 0Fh encountered at adapter PCI(00h,05h,00h)
Adapters disabled by user
Press Ctrl-C to start LSI Corp Configuration Utility....
LSI Corporation MPT boot ROM, no supported devices found!

It continues to boot and all is well. The post I found on the ubuntu forums sounds like the key, we need to point to the rom. Only issue is I don't see a way to do this from within proxmox. It would be great if a dev could let us know.
 
Last edited:
Can a dev please comment on whether we could point to the rom file or not, and if not, is there a way we could have this capability added seeing that it can be done within KVM. I appreciate the help.
 
adamb said:
Can a dev please comment on whether we could point to the rom file or not, and if not, is there a way we could have this capability added seeing that it can be done within KVM. I appreciate the help.
Click to expand...

Is there a place I can put in a feature request? I would really appreciate if a dev can comment on this.

Just to add here is what I am seeing in the logs.

Nov 14 15:58:53 vaultprox2 kernel: assign device: host bdf = 86:0:0
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0xf (was 0x100, writing 0x10b)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0xc (was 0x0, writing 0x90600000)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0x7 (was 0x4, writing 0x90300004)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0x5 (was 0x4, writing 0x90500004)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0x4 (was 0x1, writing 0xd001)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0x3 (was 0x0, writing 0x10)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: restoring config space at offset 0x1 (was 0x100000, writing 0x100043)
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: BAR 7: set to [mem 0x90504000-0x9051ffff 64bit] (PCI address [0x90504000-0x9051ffff]
Nov 14 15:58:53 vaultprox2 kernel: pci-stub 0000:86:00.0: BAR 9: set to [mem 0x90340000-0x904fffff 64bit] (PCI address [0x90340000-0x904fffff]
Nov 14 15:59:01 vaultprox2 kernel: pci-stub 0000:86:00.0: PCI INT A disabled
 
Last edited:
I'm having a similar problem trying to passthrough my LSI card to a Illumos VM. I got those errors on boot of the VM as well, but I'm not trying to boot from the HBA so I disabled booting in it & the errors went away.

My problems with Illumos not liking the passthrough LSI card still persist though. If I use a Debian VM, passthrough works fine.
 
churnd said:
I'm having a similar problem trying to passthrough my LSI card to a Illumos VM. I got those errors on boot of the VM as well, but I'm not trying to boot from the HBA so I disabled booting in it & the errors went away.

My problems with Illumos not liking the passthrough LSI card still persist though. If I use a Debian VM, passthrough works fine.
Click to expand...

I am also trying to not boot from the card. I am currently trying to pin down how to flash the card without the rom.
 
adamb said:
I am also trying to not boot from the card. I am currently trying to pin down how to flash the card without the rom.
Click to expand...

Boot to the host itself (proxmox host) to where you can see the POST & BIOS screens. You should get a screen that shows the LSI controller, from there you can hit ctrl+c to go into it's configurations. Disabling it's boot option is in the advanced settings I believe... it's there, you just have to dig for it.
 
churnd said:
Boot to the host itself (proxmox host) to where you can see the POST & BIOS screens. You should get a screen that shows the LSI controller, from there you can hit ctrl+c to go into it's configurations. Disabling it's boot option is in the advanced settings I believe... it's there, you just have to dig for it.
Click to expand...

Yep I tried that a week or so ago with no luck.

So I wiped the flash and rom from each card. Then flashed the cards with only the flash and not the bios rom. Sure enough my VM now boots as it should with the cards. I still feel the real issue is that proxmox can't point to a proper rom file but from the lack of response from the dev's there isn't much we can do.
 
adamb said:
Yep I tried that a week or so ago with no luck.

So I wiped the flash and rom from each card. Then flashed the cards with only the flash and not the bios rom. Sure enough my VM now boots as it should with the cards. I still feel the real issue is that proxmox can't point to a proper rom file but from the lack of response from the dev's there isn't much we can do.
Click to expand...

I want to add that this was on a pve 2.3 host with the latest updates. I am now trying this on a pve 3.1 host and running into very similar issues as before. I can manually add the card after the VM has booted, but the VM fails to boot if I have a "hostpci0: 04:00.00" entry. I wonder if something is up with this version of pve-qemu-kvm: 1.4-17.
 
Interesting on my cluster (pve 2.3) which is working with pci passthrough.

Nov 18 15:19:11 vaultprox2 kernel: assign device: host bdf = 86:0:0
Nov 18 15:19:11 vaultprox2 kernel: assign device: host bdf = 8b:0:0


Cluster (pve 3.1) which is not working with pci passthrough

Nov 18 15:00:39 testprox1 kernel: assign device: host bdf = 4:0:0
Nov 18 15:14:05 testprox1 kernel: assign device: host bdf = 4:0:0

The real pci address is 04:00.00 not 4:0:0
 
Just for the record: what host hardware are you using? Maybe this has to do with the errors you get. I'm using an IBM ServeRAID m1015, flashed to IT fw, redirected to a Debian Wheezy instance, working just fine. It's on an Intel DQ67SW.
 
Some news on this?

I get the same:

"Unable to load LSI Corporation MPT BIOS
MPT BIOS Fault 02h encountered at adapter PCI(00h,04h,00h)
Firmware Fault Code: 2667h"

What can i do to solve this problem?

Board is: X10SL7 from Supermirco

There are some information in this thread: http://ubuntuforums.org/showthread.php?t=2095998

but i don't know what to do with a rom file or where i can add it...

Thanks for your help!
 
mcflym said:
Some news on this?

I get the same:

"Unable to load LSI Corporation MPT BIOS
MPT BIOS Fault 02h encountered at adapter PCI(00h,04h,00h)
Firmware Fault Code: 2667h"

What can i do to solve this problem?

Board is: X10SL7 from Supermirco

There are some information in this thread: http://ubuntuforums.org/showthread.php?t=2095998

but i don't know what to do with a rom file or where i can add it...

Thanks for your help!
Click to expand...

I ended up flashing my cards so they no longer have the rom. Once that was done I was able to get around this issue.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back