Drives are not detected on sata card with marvell 88SE9230 chipset

B

bobzer

Well-Known Member
Nov 14, 2017
36
5
48
36
hi,

I'm surprised that proxmox does not reognized my sata card when every other system recognyze it (esxi/debian,...)

from my research it should be working out of the box like for the others because of the linux kernel that support it.

But i wasn't able to make it work.

what can i do ?

some info, i don't know what you need:
Code: 
# cat /usr/share/misc/pci.ids | grep -i "9230"
        9230  88SE9230 PCIe SATA 6Gb/s Controller
# /sbin/modinfo /lib/modules/*/kernel/drivers/ata/ahci.ko | egrep -i "filename|9230"
filename:       /lib/modules/4.13.4-1-pve/kernel/drivers/ata/ahci.ko
alias:          pci:v00001B4Bd00009230sv*sd*bc*sc*i*
# lsmod | grep ahci
ahci                   36864  2
libahci                32768  1 ahci

Hope you can help me thanks

BTW: i use proxmox 5.1
 
Last edited:
Hi, I think I have a similar sata card in my system.
This is the one I use:
http://www.delock.com/produkte/G_89395/merkmale.html

It works great in my Proxmox installation, worked out of the box.
I'm still on 4.4 however.
Does lspci show anything about your card?

This is what I get:
Code: 
# lspci -vvnn -s 01:00.0
01:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230] (rev 11) (prog-if 01 [AHCI 1.0])
   Subsystem: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230]
   Physical Slot: 1
   Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
   Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
   Latency: 0, Cache Line Size: 64 bytes
   Interrupt: pin A routed to IRQ 38
   Region 0: I/O ports at 3120 [size=8]
   Region 1: I/O ports at 3130 [size=4]
   Region 2: I/O ports at 3128 [size=8]
   Region 3: I/O ports at 3134 [size=4]
   Region 4: I/O ports at 3100 [size=32]
   Region 5: Memory at f0800000 (32-bit, non-prefetchable) [size=2K]
   [virtual] Expansion ROM at f0810000 [disabled] [size=64K]
   Capabilities: [40] Power Management version 3
       Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
       Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
   Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
       Address: fee0f00c  Data: 4183
   Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
       DevCap:   MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
           ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
       DevCtl:   Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
           RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
           MaxPayload 128 bytes, MaxReadReq 512 bytes
       DevSta:   CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
       LnkCap:   Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
           ClockPM- Surprise- LLActRep- BwNot-
       LnkCtl:   ASPM Disabled; RCB 64 bytes Disabled- CommClk+
           ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
       LnkSta:   Speed 5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
       DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
       DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
       LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
            Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
            Compliance De-emphasis: -6dB
       LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
            EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
   Capabilities: [e0] SATA HBA v0.0 BAR4 Offset=00000004
   Capabilities: [100 v1] Advanced Error Reporting
       UESta:   DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
       UEMsk:   DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
       UESvrt:   DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
       CESta:   RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
       CEMsk:   RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
       AERCap:   First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
   Kernel driver in use: ahci

Perhaps something is not right in your hardware configuration?
 
hi,

thanks for your help,
Code: 
# lspci -vvnn -s 03:00.0
03:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230] (rev 10) (prog-if 01 [AHCI 1.0])
        Subsystem: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230]
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR+ <PERR- INTx-
        Latency: 0, Cache Line Size: 32 bytes
        Interrupt: pin A routed to IRQ 38
        Region 0: I/O ports at c050 [size=8]
        Region 1: I/O ports at c040 [size=4]
        Region 2: I/O ports at c030 [size=8]
        Region 3: I/O ports at c020 [size=4]
        Region 4: I/O ports at c000 [size=32]
        Region 5: Memory at efd10000 (32-bit, non-prefetchable) [size=2K]
        Expansion ROM at efd00000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee00438  Data: 0000
        Capabilities: [70] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr+ UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM L0s Enabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [e0] SATA HBA v0.0 BAR4 Offset=00000004
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP+ Rollover- Timeout+ NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 0d, GenCap- CGenEn- ChkCap- ChkEn-
        Kernel driver in use: ahci
        Kernel modules: ahci

so it's really strange because it look like it get the correct driver in use.
but it doesn't see any drive, and just reboot with a live cd and everything is working...
that's strange

BTW: my card is a SYBA SD-PEX40054 PCI-Express 2.0 x2 SATA III (6.0Gb/s) HyperDuo 4-port RAID Controller Card
 
Hi, I'm now running Proxmox 5.1 without issues, still the same sata card. Fresh install.
While comparing the lspci outputs I see there are some differences between our cards, you've got a rev 10, mine is a rev11.
Also we are running on different hardware.
Searching on the net gives that many people are using this Marvell chip without issues.
The issues I did find were concerning iommu and hardware conflicts.
So my guess remains it's hardware (configuration) related.

Perhaps, just to see if it makes a difference, it's worth a try to change for example:
- bios settings: disable VTd
- put the card in another slot
- try without other pci cards
- try another SATA card

Also to make the issue more specific, I read that your sata card is recognized, but your drives aren't .
So if i'm right the head line should be:
Drives are not detected on sata card with marvell 88SE9230 chipset

Does this also mean you don't have any output from fdisk -l or lsblk?
And at what stage does this issue appear? When installing Proxmox form USB or CD or in a already installed and running Proxmox?
 
  • Like
Reactions: GadgetPig
you're correct thanks, i change the title
also the hardware config and system is working perfectly with esxi and a gparted live.
it seems that just proxmox don't work.
thanks for your help.

so with the uefi, the esxi and gparted live it's working but with proxmox only the lspci give result
lsblk and fdisk give me nothing
install from usb or running it doesnt work.
 
bobzer said:
lsblk and fdisk give me nothing
Click to expand...

That doesn't sound well.
Gparted live is based on debian, as is proxmox.
To be sure you could try latest Debian live cd/usb and verify if your drives are detected.
Proxmox kernel is based on ubuntu kernel, at the moment that's 4.13.
You could try an ubuntu 17.10 live cd/usb, perhaps more recent kernels triggered your issue.
And assuming you only tested Proxmox 5.1, you could download 4.4 and try that out, it's still supported until 2018-06.

If only running Proxmox doesn't detect the drives, you should carefully examine your log files and try to make sense whats going wrong in the boot process.
You could try dmesg or journalctl and look for errors and warnings.
If drives are detected you should see messages containing "ata" or "scsi".

In case you don't get any further perhaps a proxmox guru can help you out, my knowledge is limited too ;-)
For real support you should get the paid subscription, it's worth it!
 
Last edited:
thanks for the idea i looked at var/log/messages and discover this:

Code: 
Nov 27 13:30:48 gladys kernel: [    7.644037] ata14.00: qc timeout (cmd 0xa0)
Nov 27 13:30:48 gladys kernel: [    7.644101] ata8.00: qc timeout (cmd 0xec)
Nov 27 13:30:48 gladys kernel: [    7.900035] ata10.00: qc timeout (cmd 0xec)
Nov 27 13:30:48 gladys kernel: [    7.900060] ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [    8.144430] ata14.00: TEST_UNIT_READY failed (err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [    8.144507] ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [    8.214508] ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov 27 13:30:48 gladys kernel: [    8.958587] ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov 27 13:30:48 gladys kernel: [    8.958655] ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Nov 27 13:30:48 gladys kernel: [   14.044042] ata14.00: qc timeout (cmd 0xa1)
Nov 27 13:30:48 gladys kernel: [   14.544397] ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [   14.544412] ata14: limiting SATA link speed to 1.5 Gbps
Nov 27 13:30:48 gladys kernel: [   15.358599] ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Nov 27 13:30:48 gladys kernel: [   18.396034] ata10.00: qc timeout (cmd 0xec)
Nov 27 13:30:48 gladys kernel: [   18.396058] ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [   18.396065] ata10: limiting SATA link speed to 3.0 Gbps
Nov 27 13:30:48 gladys kernel: [   18.710592] ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Nov 27 13:30:48 gladys kernel: [   19.164033] ata8.00: qc timeout (cmd 0xec)
Nov 27 13:30:48 gladys kernel: [   19.664333] ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [   19.664340] ata8: limiting SATA link speed to 3.0 Gbps
Nov 27 13:30:48 gladys kernel: [   20.478592] ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Nov 27 13:30:48 gladys kernel: [   25.564043] ata14.00: qc timeout (cmd 0xa1)
Nov 27 13:30:48 gladys kernel: [   26.064567] ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [   26.064575] ata14.00: disabled
Nov 27 13:30:48 gladys kernel: [   26.564956] ata14: hard resetting link
Nov 27 13:30:48 gladys kernel: [   27.378600] ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Nov 27 13:30:48 gladys kernel: [   27.876768] ata14: EH complete
Nov 27 13:30:48 gladys kernel: [   50.140037] ata10.00: qc timeout (cmd 0xec)
Nov 27 13:30:48 gladys kernel: [   50.140050] ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [   50.454571] ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Nov 27 13:30:48 gladys kernel: [   52.188037] ata8.00: qc timeout (cmd 0xec)
Nov 27 13:30:48 gladys kernel: [   52.688346] ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Nov 27 13:30:48 gladys kernel: [   53.502588] ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
i guess it related but i have no idea what i can do with this

thank you very much for helping me
 
I think this bug report matches pretty well:
https://bugzilla.kernel.org/show_bug.cgi?id=42679

"our" card is mentioned several times as well as other Marvell sata cards from the 88SE9xxx range.
Comment #19 seems to summarize:
"The root cause of this bug seems to be : the device illegally accessed the memory that should be reserved for IOMMU module, and this changed iommu registers."

My guess is that you have Intel VT-d enabled in your bios. You could try to disable it, if you don't plan to pass through your hardware to your vm's.
Or you could try booting specifically with kernel parameter intel_iommu=off

There are several other suggestions in this lengthy bug report, such as a bios upgrade.
It could just be a bad implementation of VT-d on your motherboard that pops up issues in specific software/hardware combination.
Take your time to read it, apparently you are not the only one.
Perhaps also contribute your own comment to the bug report, with your full dmesg and results.

Other notes:
Have you tried the other options I proposed?
 
OK, that's fine.
I hope that one works out better for you and you get your rig working with proxmox

Perhaps other people can get some reference out of this thread when deciding on buying a Marvell based card.
 
hi,

i finally received my card and after some try i made it works.

so to make it work you have to plus in the first pcie port, that's really annoying but at least it works.

I also update the firmware but that was more in the hope that it would make it works on another pcie port ... sadly it didn't so i don't know if that was usefull at all.

anyway it's a decent card that work for me.

at least until i try to pci passthrough another card, at that point i don't yet why but it make the card disapear...
it's really strange but when i start a vm that have a pci passthrough the card and all its drives disapear...
and i have to reboot the host to make them reappear...
i'm still investigating that issue, any help would be greatly appreciate
 
janssensm said:
I think this bug report matches pretty well:
https://bugzilla.kernel.org/show_bug.cgi?id=42679

"our" card is mentioned several times as well as other Marvell sata cards from the 88SE9xxx range.
Comment #19 seems to summarize:
"The root cause of this bug seems to be : the device illegally accessed the memory that should be reserved for IOMMU module, and this changed iommu registers."

My guess is that you have Intel VT-d enabled in your bios. You could try to disable it, if you don't plan to pass through your hardware to your vm's.
Or you could try booting specifically with kernel parameter intel_iommu=off

There are several other suggestions in this lengthy bug report, such as a bios upgrade.
It could just be a bad implementation of VT-d on your motherboard that pops up issues in specific software/hardware combination.
Take your time to read it, apparently you are not the only one.
Perhaps also contribute your own comment to the bug report, with your full dmesg and results.

Other notes:
Have you tried the other options I proposed?
Click to expand...

Hi Janssensm,

Your reply helped me a lot got me on the right track.

I bought IOCrest SI-PEX50102 M.2 Plus SATA to PCI-E x2 Hardware Raid Card
Vendor: Marvell Technology
Device: 88SE9230 PCIe SATA 6Gb/s Controller
With two 1TB M.2 Cards and a 3TB HDD via SATA cable installed on to the card.

During boot I can see the card and all the drives listed, I knew it wasn't related to hardware and more of a kernel issue.
After proxmox Virtual Environment 5.3-9 loaded the drives were not present.

I followed your suggestions and after trying different combinations I was able to get the drives to show up with the following config change in /etc/default/grub without disabling VT-d in the BIOS (I need this for passthrough to my VM and LXCs):
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
GRUB_CMDLINE_LINUX="iommu=soft"

Then update grub and reboot the server and the drives are present:
sudo update-grub

Thanks again!
 
  • Like
Reactions: jtmsrl
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back