[SOLVED] Machine won't boot with 2 identical HBA IT mode card

[cl1238]

Hi, All
I recently want to use 2 HBA card so I can passthrough one for VM and one for Proxmox use.
However when I put 2 identical LSI 9300-8i (SAS3008) - which already flashed in IT mode on purchase, machine won't boot (with a blank screen). Normally it should display the HBA card & connected drives, but it showed nothing.
When I use the card individually, both works.
When I use a different HBA card (eg an Dell H200 in IT mode) with either one card, it worked.
Is there a way to get the 2 identical card worked together, I suspect due to identical firmware which may have identical id/address etc..
Any suggestion?
Thanks

 

[LnxBil]

Do both adapters have different SAS IDs? Those are normally (re)programmed at IT flash time. So you suspect the correct problem. I have no idea how to read out the sas address without the actual sas reflash utilities.

 

[cl1238]

Do both adapters have different SAS IDs? Those are normally (re)programmed at IT flash time. So you suspect the correct problem. I have no idea how to read out the sas address without the actual sas reflash utilities.

I load bios default and replug both SAS card and it did boot up. However, during the SAS bios loading, it shows an error of HBA configuration change and only show 1 HBA with its attached disks. From PVE I can see all disk although I'm not sure if all those disk will function correct.
I've tried to use 1 card at a time and observe that they both have the same SAS address (it will display during boot with the attach disk serial# etc..).
Now the question is find instruction on how to change the SAS address for 1 of the card.

 

[cl1238]

Further testing, I found that I misread the info during boot.
Both cards do have different sas address.
Maybe it is good enough to use both card as long as machine does boot up and ignore the HBA configuration change message (possible due to they both have identical version firmware etc...)
May try in future with later version of firmware from Broadcom to see if it can distinguish both card separately.

 

[LnxBil]

If the disks are all shown in PVE, you will not have a problem. Test all of them, e.g. read them with dd and check if everything works.

 

[gseeley]

Use the driverctl package to bind the vfio-pci driver to one of the cards as it allows designation by PCI bus rather than by PCI vendor ID which doesn't work with multiple cards with the same ID.
I've used this to allow one LSI HBA to be used by Proxmox and another identical HBA to be passed through to a VM.

 

[cl1238]

Use the driverctl package to bind the vfio-pci driver to one of the cards as it allows designation by PCI bus rather than by PCI vendor ID which doesn't work with multiple cards with the same ID.
I've used this to allow one LSI HBA to be used by Proxmox and another identical HBA to be passed through to a VM.

Thanks for the suggestion, I'll give it a try.

 

[cl1238]

Problem now solved.
I've downloaded the latest firmware/bios for 9300-8i, flash one of the card, machine load the later bios and correct identify both cards and their drives.
I then flash the other one to the lastest.
Now all working to the way I wanted - no more error during boot.
Firmware version 16.00.10.00, BIOS 08.37.00.00, UEFI 18.00.00.00
Thanks for all your help.