[SOLVED] PVE does not detect LSI controller connected SAS drives, pls help.

[ArturLorek]

Hello good people.
Been reading this forum for a while, as have recently started my Proxmox journey, although am quite a long term user / admin of ESXi with a small environment of few hosts.
Yet Proxmox is different and right at the beginning I hit a wall, which I am hoping the great knowledgable users will help me find a solution for.
A short description of the situation:
Hardware - NVMe Drive with Proxmox VE 7.2 installation, Fujitsu Siemens CP400i hardware controller (allowing for both JBOD as well as RAID functionality) with 4x SAS3 drives on the backplane, which - Proxmox does not see at all.
lsblk returns nothing but the NVMe. None of the drives on the controller are visible (controller itself sees them just fine).
Tried to test with a hardware raid (although not formatted, but VD created and initialised), believing that Proxmox would see VD as 1 new drive. But nothing !!
the drives are not listed in /dev/disk/by-id/ either. Would they need to be formatted with some particular fs to be seen ?? This is not required for Ununtu to recognise them OK....

lsscsi does not show them either. As if they were not there at all, but the controller sees them just fine.

One might say - an issue with the controller or the backplane. Well - now the funny thing - when the system is booted up with a Ubuntu Live USB - all drives are available and visible as they should. So it is not the hardware, it seems more Debian / Proxmox issue of not seeing the controller properly / missing drivers.
However lspci -vv returns an entry for the controller just fine, showing nicely MegaRaid LSI SAS 3008 (Fury) bla bla bla.
So it does see the controller on the pci, but no disks attached to it, although those same disks are discovered by the card itself as well as by the Ubuntu Live CD on this same very system.
Is it likely that drivers are missing ?? Since lspci is showing the card properly I assume it must have the drivers to properly identify the card. Or am I wrong here ? how do we find out if what is required by a hardware to work properly is in the system ?

Magic ?? I am out of ideas, help me some Proxmox Obi Wan Kenobi

 

[bbgeek17]

When you boot with Ubuntu disk - what kernel is it using? Get a list of modules (lsmod), is there one that you think is responsible for your controller? Get its version (modinfo). When you boot into PVE - what kernel is it using? Is the same module loaded? Is it the same version?
You may want to try to enable testing repo to see if another kernel is available.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[bbgeek17]

You could also just install appropriate Ubuntu and then install PVE as packages on top.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[ArturLorek]

When you boot with Ubuntu disk - what kernel is it using? Get a list of modules (lsmod), is there one that you think is responsible for your controller? Get its version (modinfo). When you boot into PVE - what kernel is it using? Is the same module loaded? Is it the same version?
You may want to try to enable testing repo to see if another kernel is available.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Will definitely have it checked tomorrow and shall report on my findings.
I can tell right away the PVE is the newest 7.2.1 downloaded freshly today and freshly installed. Whatever kernel it comes with.

The Ubuntu is 18.04 but do not really remember about the kernel. Will need to double check that.

 

[ArturLorek]

OK, checked few things and the findings are as per he following.
PVE has got a little newer module megaraid_sas in the kernel than that of the Ubuntu 18.04 Live CD.

PVE v 7.2.1
root@PVE:~# modinfo megaraid_sas
filename: /lib/modules/5.15.30-2-pve/kernel/drivers/scsi/megaraid/megaraid_sas.ko
description: Broadcom MegaRAID SAS Driver
author: megaraidlinux.pdl@broadcom.com
version: 07.717.02.00-rc1
license: GPL

PVE 7.1.2
root@pve:~# modinfo megaraid_sas
filename: /lib/modules/5.13.19-2-pve/kernel/drivers/scsi/megaraid/megaraid_sas.ko
description: Broadcom MegaRAID SAS Driver
author: megaraidlinux.pdl@broadcom.com
version: 07.714.04.00-rc1
license: GPL

Ubuntu 18.04
filename: /lib/modules/5.4.0-109-generic/kernel/drivers/scsi/megaraid/megaraid_sas.ko
description: Broadcom MegaRAID SAS Driver
author: megaraidlinux.pdl@broadcom.com
version: 07.713.01.00-rc1
license: GPL

Kernel version difference is signifficant, but somehow there is something else.
I checked journal ctl on proxmox and it returned:
May 06 12:57:05 pve kernel: megaraid_sas 0000:01:00.0: Init cmd return status FAILED for SCSI host 0
May 06 12:57:05 pve kernel: megaraid_sas 0000:01:00.0: Failed from megasas_init_fw 6540

This would make sense as to why lsmod on PVE returned:

root@pve:~# lsmod
Module Size Used by
megaraid_sas 163840 0

It is not running and not claimed by any device.

So the question now is - what to do to get it running and why did it fail on Proxmox / Debian bootup and it did not on Ubuntu Live CD ??

 

[bbgeek17]

Sounds like potential compatibility issue between card firmware and module. Things get broken all the time in most unexpected place. Make sure you have latest firmware installed, reach out to HP for support if possible.
One thing is clear - its a software/firmware issue outside of PVE customization. I am confident they make no changes to "MegaRAID SAS Driver".

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[ArturLorek]

SOLVED.
Trivial. This was to do with the BIOS settings> CSM > storage

IT HAS TO BE UEFI ONLY.

Not Uefi or Legacy and deffinitely not Legacy ONLY.
Just UEFI and all the disks are now visible as the controller is properly detected. How could I not have seen it before.
Thank you for your help bbgeek17