[SOLVED] proxmox 7 / linux 5.11.22 issue with LSI 2008 controllers?

C

crabbypup

Member
Aug 2, 2021
23
3
8
32
EDIT: if you are using external enclosures with LSI 2008 based sas cards, you need to add
Code: 
mpt3sas.max_queue_depth=10000
To your kernel boot line in /etc/default/grub or /etc/kernel/cmdline if you are on an efi platform and aren't using efi-grub

Hi, I believe I may have found an issue with Proxmox 7, though more specifically how some LSI cards are handled by the mpt3sas module present in kernels from version 5.11.20 and onward.


Hardware:
Supermicro H8DG6-f board
Dual AMD opteron 6276
64GB ddr3 ECC reg
Mellanox connect-x2 10g nic
Renesas/NEC USB 3.0 controller
AMD Radeon RX580
Micron P320h pcie ssd
boot volume is raidz2 of 4x 1tb 2.5" disks off the onboard sata controller
LSI 9200-8e HBA
2x Supermicro 12 bay SAS2 expander backplanes
mix of 16x 2TB NL-SAS and SATA 7200RPM drives on the expander backplanes

The 9200-8e works fine under distros with kernels up to 5.11.12 (tested with a fedora 34 live disk). But on at least proxmox 7 with 5.11.22, the 9200-8e fails to initialize with the following dmesg printout (only lines relating to mpt2sas and mpt3sas selected):

Code: 
[    3.985411] mpt3sas version 36.100.00.00 loaded
[    3.985615] mpt3sas 0000:41:00.0: can't disable ASPM; OS doesn't have ASPM control
[    4.022225] mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65787168 kB)
[    4.022255] mpt2sas_cm0: sending diag reset !!
....
[    4.188931] mpt2sas_cm0: Invalid host diagnostic register value
[    4.188938] mpt2sas_cm0: System Register set:
[    4.188940] 00000000: ffffffff
[    4.188943] 00000004: ffffffff
[    4.188944] 00000008: ffffffff
[    4.188946] 0000000c: ffffffff
[    4.188948] 00000010: ffffffff
[    4.188949] 00000014: ffffffff
[    4.188951] 00000018: ffffffff
[    4.188952] 0000001c: ffffffff
[    4.188954] 00000020: ffffffff
[    4.188955] 00000024: ffffffff
[    4.188957] 00000028: ffffffff
[    4.188958] 0000002c: ffffffff
[    4.188960] 00000030: ffffffff
[    4.188961] 00000034: ffffffff
[    4.188962] 00000038: ffffffff
[    4.188964] 0000003c: ffffffff
[    4.188965] 00000040: ffffffff
[    4.188966] 00000044: ffffffff
[    4.188968] 00000048: ffffffff
[    4.188969] 0000004c: ffffffff
[    4.188971] 00000050: ffffffff
[    4.188972] 00000054: ffffffff
[    4.188974] 00000058: ffffffff
[    4.188975] 0000005c: ffffffff
[    4.188976] 00000060: ffffffff
[    4.188978] 00000064: ffffffff
[    4.188979] 00000068: ffffffff
[    4.188981] 0000006c: ffffffff
[    4.188982] 00000070: ffffffff
[    4.188984] 00000074: ffffffff
[    4.188985] 00000078: ffffffff
....
[    4.189052] 000000ec: ffffffff
[    4.189053] 000000f0: ffffffff
[    4.189055] 000000f4: ffffffff
[    4.189056] 000000f8: ffffffff
[    4.189058] 000000fc: ffffffff
[    4.189060] mpt2sas_cm0: diag reset: FAILED
[    4.189262] mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:12013/_scsih_probe()!
....
[   17.886801] Fusion MPT base driver 3.04.20
[   17.886807] Copyright (c) 1999-2008 LSI Corporation
[   17.888205] Fusion MPT misc device (ioctl) driver 3.04.20
[   17.888297] mptctl: Registered with Fusion MPT base driver
[   17.888299] mptctl: /dev/mptctl @ (major,minor=10,220)

And obviously none of the disks attached to the controller show up after boot.

My motherboard is on the latest available bios, and the 9200-8e is on the latest available firmware (P20). The proxmox kernel is up to date, 5.11.22-3 as of writing.
The controller sees the drives just fine, if I go into the controller bios during boot, all drives and backplanes are listed in the topology listing.

Is there anyone else out there experiencing this problem with any other LSI SAS2008 based HBAs under proxmox 7?
Is this specific to the 9200-8e?

Is this actually a bug, or is it something in my specific hardware that's not behaving?
Though if it is, it's rather odd that the controller would be fine under earlier kernels if it is hardware weirdness.

As alluded to, if this is a bug, I believe it would have been introduced in 5.11.20 - possibly in either of these two commits, though I could definitely be wrong, but they are among the very few changes to the mpt3sas module between 5.11.12 and 5.11.22
https://git.kernel.org/pub/scm/linu...y&id=fd5c036724c370c25c24918523d9d38cbe0cbe4a
https://git.kernel.org/pub/scm/linu...y&id=828aa1d87f83820459f0adc6182c027cd907c4ce

Thanks everyone.
 
Last edited:
  • Like
Reactions: easyronny
@fabian

Im not very technical if required can test it with proxmox version 7.01.
What are exactly the steps that I need to execute that is not entirely clear to me.

Ronny
 
http://download.proxmox.com/temp/pve-kernel-mpt3sas-test/

Code: 
aaf80a860db892a00b9a4064b0516107dd43f1bc05377311da05a869132a8727  linux-tools-5.11_5.11.22-7~test2_amd64.deb
44112057cd8c7a276694eab345be8fc0f9a7965515a23d26cc24c7795b72feb0  linux-tools-5.11-dbgsym_5.11.22-7~test2_amd64.deb
3e78ab10ecaf48b9ec970dae045fb9f5bb09e8e51c2ac0d4464cfdfacfca202b  pve-headers-5.11.22-3-pve_5.11.22-7~test2_amd64.deb
1b79bbb43d124e7b82cd65aed235458b51a197ecc8aaab96342c7e0e3e065c1b  pve-kernel-5.11.22-3-pve_5.11.22-7~test2_amd64.deb
5254a55690d566c6c92d195b25ea199901be9850bc884a17de01fc01dfa39be6  pve-kernel-libc-dev_5.11.22-7~test2_amd64.deb

contains patches 2 and 3 from the series I linked. if that does not help, I can also build one with the commits you linked reverted individually. to get back to the stock kernel, run apt install --reinstall $PACKAGE=5.11.22-6 for all $PACKAGE values that you installed manually (e.g., pve-kernel-5.11.22-3-pve)
 
  • Like
Reactions: crabbypup and easyronny
@fabian and orthers,

I downloaded the files and copyed the files towards the home folder of proxmox.
The command APT install --reinstall did not work for me

The following commands are executed by my and as far as my knowledge goes (im not an expert)

dpkg -i linux-tools-5.11_5.11.22-7_test2_amd64.deb
dpkg -i linux-tools-5.11-dbgsym_5.11.22-7~test2_amd64.deb
dpkg -i linux-tools-5.11_5.11.22-7~test2_amd64.deb
dpkg -i pve-headers-5.11.22-3-pve_5.11.22-7~test2_amd64.deb
dpkg -i pve-kernel-5.11.22-3-pve_5.11.22-7~test2_amd64.deb
dpkg -i pve-kernel-libc-dev_5.11.22-7~test2_amd64.deb

proxmox-boot-tool refresh
update-initramfs -u

See my attachment with the errors that I get on the console screen at the moment when I started a virtual machine (now it fits on the screen).


Ronny
 

Attachments

  • 2021_08_02_16_48_55_Photos-SM.jpg
    2021_08_02_16_48_55_Photos-SM.jpg
    396.9 KB · Views: 49
Ok, I have some more testing. I booted from a siduction 21.2.0 live disk, which uses kernel 5.13.6 (deb unstable, as the name implies), and the card initialises properly.

more importantly, it does not issue a diagnostic reset to the card at any time.
 
I'm working to narrow down what the exact issue I'm experiencing is.

My brother has a Dell r710 with a perc H200 (basically an LSI 9211-8i with some special-ish hybrid IR/IT firmware - his is not crossflashed), and using the install media for proxmox 7.0, his card initialized fine. I went back to test if my 9200-8e worked under the install media -1 kernel, and it does not work.

I also tried the fix documented here for an issue that was present in 5.8, thinking there may have been a regression, as one of the dmesg lines I'm getting is similar, but there was no change: https://www.reddit.com/r/homelab/co...work/?utm_medium=android_app&utm_source=share

So it appears something possibly specific to cards with IT firmware, or just this specific model of LSI card, but that derivative branded cards like the M1015, H200, H310, etc - are likely unaffected.

At this point, it might be useful for someone with an H200 that was flashed with 9211 IT firmware to test this, and help narrow down where specifically the issue is that's causing the module to call for the diag_reset.

I've also been looking at commits to the mpt3sas module in the 5.13.y tree. There does seem to be a large merge here:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.13.y&id=d72cd4ad4174cfd2257c426ad51e4f53bcfde9c9

Which has significant changes to the mpt3sas module. Despite the commit message listed in the header, there's a fair bit to that rollup, some of which automatically backs down the queue depth as necessary. That is rather early in the 5.13 dev cycle (pre-rc1) however, it was not applied to the 5.11.22 tree from what I can tell.

Possibly stupid question, but could the mpt3sas module from 5.13.6 be backported to 5.11.22? or, perhaps more achievable, could the module in 5.11.22 that is currently in use be rolled back to the state it was in 5.11.12?
 
Last edited:
As a datapoint:

I just completed a new 5-node builld, with several sets of 92xx cards
* AOC-USAS-L8i (Broadcom 1068E)
* LSI 9207-8i (IBM M5110)
* AOC-S2308L-L8E (LSI 9207-8i)
* two other random LSI 92xx cards

I tried the cards as they were, and cross-flashed them to v20 (as appropriate), and none of those 3 cards have issues under 7.x with latest kernels, so narrowing this down the H200 seems like you're on the right path.
 
  • Like
Reactions: crabbypup
another test kernel, this is with https://git.kernel.org/pub/scm/linu...y&id=828aa1d87f83820459f0adc6182c027cd907c4ce reverted

Code: 
f22a4b50b182fb2c07bfbfec4f19893b038bef5ff61f39c13fa97336bb9caa7c  linux-tools-5.11_5.11.22-7~test3_amd64.deb
c167df7435954506695c7b00bd0eadd1005da043f3e5558ccd54b5dc95872654  linux-tools-5.11-dbgsym_5.11.22-7~test3_amd64.deb
0e49e3b51bc45a323652efb5438c09bbdecf7caf090bb9ec13d7496f0a82e3dc  pve-headers-5.11.22-3-pve_5.11.22-7~test3_amd64.deb
4965fc707f07447be252fad1d243196a0ab19c9c880fe83efe7e0b935aa86ce5  pve-kernel-5.11.22-3-pve_5.11.22-7~test3_amd64.deb
1b1ee7fa6c9d2497bca12e7e95f66fb646de3920e4c31c87d1b96a70e1ae94fa  pve-kernel-libc-dev_5.11.22-7~test3_amd64.deb
 
  • Like
Reactions: crabbypup
Thanks Fabian, I'm not seeing any change with that kernel either.

As a note, I'll be away from my systems for a bit and won't be able to test further until the 13th.
 
feel free to ping here if the issue is not resolved with the then current kernels and well see about further bisecting this.
 
So, I'm back.

I found a workaround by using the pve-edge 5.13.x (currently 5.13.9) kernel.

Given the lack of other people experiencing this specific issue, and that others with similar cards using the same controller are not experiencing this issue, I'm personally ok with this workaround, with full acknowledgement that it is definitely not production ready.

If anyone else comes forward with this issue, I'd be happy to help troubleshoot further, but for now my system is working well with the workaround.
 
Hi, just to add another datapoint as @dlasher did.
I have a Dell H310 PERC flashed to IT mode with FW P20.
It currently runs most recent PVE 7 with no-subscription repo with kernel 5.11.22-3-pve.
And it runs just fine, as it did with previous pve 6 kernels.

You posted just a small part of your log regarding the sas controller. My log shows a lot more, f.e. the firmware version and protocol used.
Maybe there are more interesting messages in your log, could you post the full log or something like journalctl -b | grep -i sas ?
Perhaps you have hit a special corner case because you are using expander backplanes.
 
  • Like
Reactions: crabbypup
janssensm said:
Hi, just to add another datapoint as @dlasher did.
I have a Dell H310 PERC flashed to IT mode with FW P20.
It currently runs most recent PVE 7 with no-subscription repo with kernel 5.11.22-3-pve.
And it runs just fine, as it did with previous pve 6 kernels.

You posted just a small part of your log regarding the sas controller. My log shows a lot more, f.e. the firmware version and protocol used.
Maybe there are more interesting messages in your log, could you post the full log or something like journalctl -b | grep -i sas ?
Perhaps you have hit a special corner case because you are using expander backplanes.
Click to expand...

Unfortunately, that's not likely to be the case - or at least there is little evidence to suggest it is so.
Even with my DAS/backplanes powered down, the module still was failing to load.

Code: 
### First boot ever recorded by journalctl, post install. DAS was not powered on at this time.

# journalctl -b f42de98162c54969aa10ed6f43f85e72 | egrep -e '(Linux version|sas|SAS|mptctl)'
Aug 01 19:16:29 Atlas kernel: Linux version 5.11.22-1-pve (build@proxmox) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.11.22-2 (Fri, 02 Jul 2021 16:22:45 +0200) ()
Aug 01 19:16:29 Atlas kernel: mpt3sas version 36.100.00.00 loaded
Aug 01 19:16:29 Atlas kernel: mpt3sas 0000:41:00.0: can't disable ASPM; OS doesn't have ASPM control
Aug 01 19:16:29 Atlas kernel: mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65853100 kB)
Aug 01 19:16:29 Atlas kernel: mpt2sas_cm0: sending diag reset !!
Aug 01 19:16:29 Atlas kernel: mpt2sas_cm0: Invalid host diagnostic register value
Aug 01 19:16:29 Atlas kernel: mpt2sas_cm0: System Register set:
Aug 01 19:16:29 Atlas kernel: mpt2sas_cm0: diag reset: FAILED
Aug 01 19:16:29 Atlas kernel: mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:12013/_scsih_probe()!
Aug 01 19:16:37 Atlas smartd[1857]: Monitoring 4 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices

### From a later boot, just before I started this thread
# journalctl -b 4774cf8aea6a4342bf235373196f0754 | egrep -e '(Linux version|sas|SAS|mptctl)'
Aug 02 01:52:53 Atlas kernel: Linux version 5.11.22-3-pve (build@proxmox) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.11.22-6 (Wed, 28 Jul 2021 10:51:12 +0200) ()
Aug 02 01:52:53 Atlas kernel: mpt3sas version 36.100.00.00 loaded
Aug 02 01:52:53 Atlas kernel: mpt3sas 0000:41:00.0: can't disable ASPM; OS doesn't have ASPM control
Aug 02 01:52:53 Atlas kernel: mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65787168 kB)
Aug 02 01:52:53 Atlas kernel: mpt2sas_cm0: sending diag reset !!
Aug 02 01:52:53 Atlas kernel: mpt2sas_cm0: Invalid host diagnostic register value
Aug 02 01:52:53 Atlas kernel: mpt2sas_cm0: System Register set:
Aug 02 01:52:53 Atlas kernel: mpt2sas_cm0: diag reset: FAILED
Aug 02 01:52:53 Atlas kernel: mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:12013/_scsih_probe()!
Aug 02 01:52:58 Atlas smartd[2844]: Monitoring 4 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices
Aug 02 01:53:01 Atlas kernel: mptctl: Registered with Fusion MPT base driver
Aug 02 01:53:01 Atlas kernel: mptctl: /dev/mptctl @ (major,minor=10,220)

### First test kernel
# journalctl -b f8120ac4ba9640a0b6ba111617a0e2f3 | egrep -e '(Linux version|sas|SAS|mptctl)'
Aug 02 18:43:48 Atlas kernel: Linux version 5.11.22-3-pve (fgruenbichler@nora) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.11.22-7~test2 (Mon, 2 Aug 2021 13:55:04 +0200) ()
Aug 02 18:43:48 Atlas kernel: mpt3sas version 36.100.00.00 loaded
Aug 02 18:43:48 Atlas kernel: mpt3sas 0000:41:00.0: can't disable ASPM; OS doesn't have ASPM control
Aug 02 18:43:48 Atlas kernel: mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65787176 kB)
Aug 02 18:43:48 Atlas kernel: mpt2sas_cm0: sending diag reset !!
Aug 02 18:43:48 Atlas kernel: mpt2sas_cm0: Invalid host diagnostic register value
Aug 02 18:43:48 Atlas kernel: mpt2sas_cm0: System Register set:
Aug 02 18:43:48 Atlas kernel: mpt2sas_cm0: diag reset: FAILED
Aug 02 18:43:48 Atlas kernel: mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:12141/_scsih_probe()!
Aug 02 18:43:53 Atlas smartd[2311]: Monitoring 4 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices
Aug 02 18:44:00 Atlas kernel: mptctl: Registered with Fusion MPT base driver
Aug 02 18:44:00 Atlas kernel: mptctl: /dev/mptctl @ (major,minor=10,220)

### Second test kernel
# journalctl -b f5ff192c6bea463f99a9925cbcca7b5d | egrep -e '(Linux version|sas|SAS|mptctl)'
Aug 03 09:16:58 Atlas kernel: Linux version 5.11.22-3-pve (fgruenbichler@nora) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.11.22-7~test3 (Mon, 2 Aug 2021 16:40:19 +0200) ()
Aug 03 09:16:58 Atlas kernel: Command line: BOOT_IMAGE=/vmlinuz-5.11.22-3-pve root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt mpt3sas.max_queue_depth=8000 quiet
Aug 03 09:16:58 Atlas kernel: Kernel command line: BOOT_IMAGE=/vmlinuz-5.11.22-3-pve root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt mpt3sas.max_queue_depth=8000 quiet
Aug 03 09:16:58 Atlas kernel: mpt3sas version 36.100.00.00 loaded
Aug 03 09:16:58 Atlas kernel: mpt3sas 0000:41:00.0: can't disable ASPM; OS doesn't have ASPM control
Aug 03 09:16:58 Atlas kernel: mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65787176 kB)
Aug 03 09:16:58 Atlas kernel: mpt2sas_cm0: sending diag reset !!
Aug 03 09:16:58 Atlas kernel: mpt2sas_cm0: Invalid host diagnostic register value
Aug 03 09:16:58 Atlas kernel: mpt2sas_cm0: System Register set:
Aug 03 09:16:58 Atlas kernel: mpt2sas_cm0: diag reset: FAILED
Aug 03 09:16:58 Atlas kernel: mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:12013/_scsih_probe()!
Aug 03 09:17:05 Atlas smartd[5307]: Monitoring 4 ATA/SATA, 0 SCSI/SAS and 0 NVMe devices
Aug 03 09:17:13 Atlas kernel: mptctl: Registered with Fusion MPT base driver
Aug 03 09:17:13 Atlas kernel: mptctl: /dev/mptctl @ (major,minor=10,220)
 
Last edited:
For comparison, the boot log with the edge kernel:

Code: 
### Edge kernel
# journalctl -b 10e72e848a5c4a9fb2abdbd5901e353a | egrep -e '(Linux version|sas|SAS|mptctl)'
Aug 14 17:16:58 Atlas kernel: Linux version 5.13.9-1-edge (github@pve-bullseye) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP 5.13.9-1-edge generic (Sun, 08 Aug 2021 12:10:15 +0000)
Aug 14 17:16:58 Atlas kernel: Command line: BOOT_IMAGE=/vmlinuz-5.13.9-1-edge root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt mpt3sas.max_queue_depth=8000 quiet
Aug 14 17:16:58 Atlas kernel: Kernel command line: BOOT_IMAGE=/vmlinuz-5.13.9-1-edge root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt mpt3sas.max_queue_depth=8000 quiet
Aug 14 17:16:58 Atlas kernel: mpt3sas version 37.101.00.00 loaded
Aug 14 17:16:58 Atlas kernel: mpt3sas 0000:41:00.0: can't disable ASPM; OS doesn't have ASPM control
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65790812 kB)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 4k
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: MSI-X vectors supported: 1
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0:  0 1
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: High IOPs queues : disabled
Aug 14 17:16:58 Atlas kernel: mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 52
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: iomem(0x00000000bec3c000), mapped(0x(____ptrval____)), size(16384)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: ioport(0x000000000000e000), size(256)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 4k
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: sending message unit reset !!
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: message unit reset: SUCCESS
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: scatter gather: sge_in_main_msg(1), sge_per_chain(9), sge_per_io(128), chains_per_io(15)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: request pool(0x(____ptrval____)) - dma(0x112900000): depth(2124), frame_size(128), pool_size(265 kB)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: sense pool(0x(____ptrval____)) - dma(0x112cc0000): depth(1951), element_size(96), pool_size (182 kB)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: sense pool(0x(____ptrval____))- dma(0x112cc0000): depth(1951),element_size(96), pool_size(0 kB)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: reply pool(0x(____ptrval____)) - dma(0x112d00000): depth(2188), frame_size(128), pool_size(273 kB)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: config page(0x(____ptrval____)) - dma(0x112c91000): size(512)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: Allocated physical memory: size(4422 kB)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: Current Controller Queue Depth(1948),Max Controller Queue Depth(2040)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: Scatter Gather Elements per IO(128)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: LSISAS2008: FWVersion(20.00.07.00), ChipRevision(0x03), BiosVersion(07.39.02.00)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
Aug 14 17:16:58 Atlas kernel: scsi host0: Fusion MPT SAS Host
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: sending port enable !!
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: hba_port entry: (____ptrval____), port: 255 is added to hba_port list
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: host_add: handle(0x0001), sas_addr(0x500605b007f0a8d0), phys(8)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x5003048001dce17f), phys(30)
Aug 14 17:16:58 Atlas kernel:  expander-0:0: add: handle(0x0009), sas_addr(0x5003048001dce17f)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: expander_add: handle(0x0013), parent(0x0002), sas_addr(0x500304800123dd7f), phys(30)
Aug 14 17:16:58 Atlas kernel:  expander-0:1: add: handle(0x0013), sas_addr(0x500304800123dd7f)
Aug 14 17:16:58 Atlas kernel: mpt2sas_cm0: port enable: SUCCESS
Aug 14 17:16:58 Atlas kernel: scsi 0:0:0:0: SATA: handle(0x0017), sas_addr(0x500304800123dd72), phy(18), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:0: add: handle(0x0017), sas_addr(0x500304800123dd72)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:1:0: SATA: handle(0x000a), sas_addr(0x5003048001dce16f), phy(15), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:0: add: handle(0x000a), sas_addr(0x5003048001dce16f)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:2:0: SATA: handle(0x000b), sas_addr(0x5003048001dce170), phy(16), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:1: add: handle(0x000b), sas_addr(0x5003048001dce170)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:3:0: SATA: handle(0x000c), sas_addr(0x5003048001dce171), phy(17), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:2: add: handle(0x000c), sas_addr(0x5003048001dce171)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:4:0: SSP: handle(0x000d), sas_addr(0x5000cca01c785b41), phy(18), device_name(0x5000cca01c785b41)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:3: add: handle(0x000d), sas_addr(0x5000cca01c785b41)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:5:0: SSP: handle(0x000e), sas_addr(0x5000cca01c6b279d), phy(19), device_name(0x5000cca01c6b279d)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:4: add: handle(0x000e), sas_addr(0x5000cca01c6b279d)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:6:0: SSP: handle(0x000f), sas_addr(0x5000cca01c7503c5), phy(20), device_name(0x5000cca01c7503c5)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:5: add: handle(0x000f), sas_addr(0x5000cca01c7503c5)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:7:0: SATA: handle(0x0010), sas_addr(0x5003048001dce175), phy(21), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:6: add: handle(0x0010), sas_addr(0x5003048001dce175)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:8:0: SSP: handle(0x0011), sas_addr(0x5000cca01c785a09), phy(23), device_name(0x5000cca01c785a09)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:7: add: handle(0x0011), sas_addr(0x5000cca01c785a09)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:9:0: Enclosure         LSI CORP SAS2X28          0717 PQ: 0 ANSI: 5
Aug 14 17:16:58 Atlas kernel: scsi 0:0:9:0: SES: handle(0x0012), sas_addr(0x5003048001dce17d), phy(28), device_name(0x5003048001dce17d)
Aug 14 17:16:58 Atlas kernel:  end_device-0:0:8: add: handle(0x0012), sas_addr(0x5003048001dce17d)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:10:0: SATA: handle(0x0014), sas_addr(0x500304800123dd6f), phy(15), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:1: add: handle(0x0014), sas_addr(0x500304800123dd6f)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:11:0: SATA: handle(0x0015), sas_addr(0x500304800123dd70), phy(16), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:2: add: handle(0x0015), sas_addr(0x500304800123dd70)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:12:0: SATA: handle(0x0016), sas_addr(0x500304800123dd71), phy(17), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:3: add: handle(0x0016), sas_addr(0x500304800123dd71)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:13:0: SATA: handle(0x0018), sas_addr(0x500304800123dd73), phy(19), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:4: add: handle(0x0018), sas_addr(0x500304800123dd73)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:14:0: SATA: handle(0x0019), sas_addr(0x500304800123dd74), phy(20), device_name(0x0000000000000000)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:5: add: handle(0x0019), sas_addr(0x500304800123dd74)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:15:0: SSP: handle(0x001a), sas_addr(0x5000cca01c77d521), phy(21), device_name(0x5000cca01c77d521)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:6: add: handle(0x001a), sas_addr(0x5000cca01c77d521)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:16:0: SSP: handle(0x001b), sas_addr(0x5000cca01c785fc9), phy(22), device_name(0x5000cca01c785fc9)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:7: add: handle(0x001b), sas_addr(0x5000cca01c785fc9)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:17:0: SSP: handle(0x001c), sas_addr(0x5000cca01c727485), phy(23), device_name(0x5000cca01c727485)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:8: add: handle(0x001c), sas_addr(0x5000cca01c727485)
Aug 14 17:16:58 Atlas kernel: scsi 0:0:18:0: Enclosure         LSI CORP SAS2X28          0717 PQ: 0 ANSI: 5
Aug 14 17:16:58 Atlas kernel: scsi 0:0:18:0: SES: handle(0x001d), sas_addr(0x500304800123dd7d), phy(28), device_name(0x500304800123dd7d)
Aug 14 17:16:58 Atlas kernel:  end_device-0:1:9: add: handle(0x001d), sas_addr(0x500304800123dd7d)
Aug 14 17:18:18 Atlas smartd[22179]: Monitoring 14 ATA/SATA, 7 SCSI/SAS and 0 NVMe devices
Aug 14 17:18:19 Atlas kernel: mptctl: Registered with Fusion MPT base driver
Aug 14 17:18:19 Atlas kernel: mptctl: /dev/mptctl @ (major,minor=10,220)
 
  • Like
Reactions: janssensm
Hello everyone ... Thanks for the efforts to get this issue resolved. I created an account just to add my .02.

I have been unable to get any hard drives attached to my: LSI Logic Controller Card H5-25379-00 SAS 9201-16e 16Port 6Gb/s SATA+SAS PCI Express
recognized by Proxmox. Drives are within a 15-bay external SAS/SATA enclosure. The card has been flashed to the latest firmware (20.07) and is in IT mode. When I boot the machine the card initializes and any attached drives are seen. The card itself is also seen by ProxMox as an available PCI passthrough device. Passthrough also works as I have tested this with UnRaid which sees any drive I throw at it. Based on this I believe I can confirm this is an issue with the kernel.

I have tried the latest Edge 5.13 kernel and in my case I had no luck.

Just put me down as affected by this and hopefully the specs of the card will help. If I can offer any logs just hit me up and I'll be happy to provide them. I am however traveling quite a bit so my reply's will mainly be Friday-Sunday.

Thanks again!
 
Cr4fty said:
Hello everyone ... Thanks for the efforts to get this issue resolved. I created an account just to add my .02.

I have been unable to get any hard drives attached to my: LSI Logic Controller Card H5-25379-00 SAS 9201-16e 16Port 6Gb/s SATA+SAS PCI Express
recognized by Proxmox. Drives are within a 15-bay external SAS/SATA enclosure. The card has been flashed to the latest firmware (20.07) and is in IT mode. When I boot the machine the card initializes and any attached drives are seen. The card itself is also seen by ProxMox as an available PCI passthrough device. Passthrough also works as I have tested this with UnRaid which sees any drive I throw at it. Based on this I believe I can confirm this is an issue with the kernel.

I have tried the latest Edge 5.13 kernel and in my case I had no luck.

Just put me down as affected by this and hopefully the specs of the card will help. If I can offer any logs just hit me up and I'll be happy to provide them. I am however traveling quite a bit so my reply's will mainly be Friday-Sunday.

Thanks again!
Click to expand...
Hey, can you post the relevant parts of your dmesg output (with passthrough disabled for the card)? It sounds like your issue might be slightly different if the edge kernel isn't helping, but might be related.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back