[SOLVED] Proxmox 7 on HP G7 and how to enable hba mode P410 (hpsahba)

E

emanuelebruno

Renowned Member
May 1, 2012
143
7
83
Catania
emanuelebruno.it
Hi all, I hope I'm not off topic talking about this. I am making a small guide that explains all steps to install the latest version of Proxmox in an HP DL 380 G7 server and how to enable HBA mode on the P410i SCSI controller. Unfortunately I am having some difficulties compiling the kernel. Therefore, I have thought about posting the guide although it is not complete in the hope that someone can tell me where I went wrong.

I would like to install Proxmox in an SSD connected to the AHCI controller of the server, enable the HBA mode of the RAID P410 controller, and be able to directly access (HBA-MODE) on all the drives connected to the controller.

To succeed in this I bought a mini-sata cable and an adapter to add an ssd hard disk instead of the DVD player provided in the server.
EW7003_0_pr.jpegImmagine 2022-02-21 100023.png

The first difficulty I encountered was the incompatibility of the latest kernel with this hardware (the system does not always boot without errors). For this reason I needed to disable the "hpwdt" driver.

Also trying to install the latest ISO (7.1-2) from pendrive, the server gives errors on "GRUB".

First you need to install Debian (https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-11.2.0-amd64-netinst.iso).

At the end of the installation you need to log in as root and type the following:
echo "blacklist hpwdt" >> /etc/modprobe.d/blacklist-hp.conf cat /etc/modprobe.d/blacklist-hp.conf
blacklist hpwdt
update-initramfs -k all -u update-grub apt update -y && apt install wget sudo htop iotop sdparm screen -y nano /etc/hosts 127.0.0.1 localhost
192.168.1.2 prox4m1.proxmox.com prox4m1→ to be adapted according to your needs
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
echo "deb [arch=amd64] http://download.proxmox.com/debian/pve bullseye pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list wget https://enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg apt update && apt full-upgrade -y apt install proxmox-ve postfix open-iscsi -y apt remove os-prober apt install curl -y curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list > /dev/null apt update && apt install gh -y cd /root git clone https://github.com/im-0/hpsahba.git cd hpsahba apt install pandoc -y make
pandoc --from markdown --to man --standalone --metadata "title=HPSAHBA(8)" --output hpsahba.8 README.md
ls /lib/modules/5.13.19-4-pve/build→ check if this folder is present on your system or adapt according to your needs
cd ~/hpsahba/contrib/dkms cp Makefile Makefile.original nano Makefile
obj-m := hpsa.o
ifndef KERNELRELEASE
KRELEASE := $(shell uname -r)
else
KRELEASE := $(KERNELRELEASE)
endif
#KDIR := /usr/lib/modules/$(KRELEASE)/build
KDIR := /lib/modules/5.13.19-4-pve/build → check if this folder is present on your system or adapt according to your needs
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) M=$(PWD) modules
clean:
$(MAKE) -C $(KDIR) M=$(PWD) clean
nano patch.sh
#!/bin/bash
set -e
#VERSION=${1:-5.8}
VERSION=${1:-$(uname -r | grep -o "^[0-9]*[.][0-9]*")} → Add this code to prevent errors
echo "Patching for kernel ${VERSION}"
wget "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/drivers/scsi/hpsa.h?h=linux-${VERSION}.y" -O hpsa.h
wget "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/drivers/scsi/hpsa.c?h=linux-${VERSION}.y" -O hpsa.c
wget "https://git.kernel.org/pub/scm/linu...x.git/plain/drivers/scsi/hpsa_cmd.h?h=linux-${VERSION}.y" -O hpsa_cmd>
shopt -s nullglob
for PATCH in ../../kernel/"${VERSION}"*/*.patch; do
echo "Applying ${PATCH}"
patch --no-backup-if-mismatch -Np3 < "${PATCH}"
done
./patch.sh
[...]→ here I avoid copying and pasting because the output is too long
dkms add ./
Creating symlink /var/lib/dkms/hpsa-dkms/1.0/source → /usr/src/hpsa-dkms-1.0
DKMS: add completed.
dkms install --force hpsa-dkms/1.0
Kernel preparation unnecessary for this kernel. Skipping...
Building module:
cleaning build area...
make -j12 KERNELRELEASE=5.13.19-4-pve....
cleaning build area...
DKMS: build completed.
hpsa.ko:
Running module version sanity check.
- Original module
- Installation
- Installing to /lib/modules/5.13.19-4-pve/updates/dkms/
depmod....
DKMS: install completed.
modprobe -r hpsa modprobe hpsa hpsa_use_nvram_hba_flag=1 echo "options hpsa hpsa_use_nvram_hba_flag=1" > /etc/modprobe.d/hpsa.co update-initramfs -u
update-initramfs: Generating /boot/initrd.img-5.13.19-4-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
init 6

***AFTER REBOOT***
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 29,8G 0 disk → This is the drive where I put Proxmox...
└─sda1 8:1 0 29,8G 0 part /
sdb 8:16 0 465,8G 0 disk → This HDD is attached to P410i controller for testing purpose
├─sdb1 8:17 0 499M 0 part
├─sdb2 8:18 0 300M 0 part
├─sdb3 8:19 0 128M 0 part
├─sdb4 8:20 0 439,4G 0 part
├─sdb5 8:21 0 449M 0 part
├─sdb6 8:22 0 24,1G 0 part
└─sdb7 8:23 0 1G 0 part
.:The End:.
 
Last edited:
  • Like
Reactions: wbk, bootsie123, kuroiookami and 2 others
HPSAHBA works great with Ubuntu 18.04 and kernel 5.40. I can see all SAS and SATA drives. I have tried probably 20 different models. No fan problems ever. Booting directly to NVME using PCI adapter with SATA connection to the DVD drive. Remember to upgrade all firmware first to latest (P67 - 08/2015).

Tried compiling patches directly into PVE-Edge 5.12, but did not work. Compiling gave no errors, but I'm new to this so I moved on to using Debian Bullseye first before tackling PVE.

I'm just trying the same as you - Debian Bullseye on a DL 380 G7. I would suggest trying kernel 5.12 which has the patch for HPSAHBA. The other "patches" are symlinks to either 5.40 or 5.12 and as far as I can tell may not work. Upgrade the latest Debian from 5.10 to 5.12 and then install HPSAHBA.

Would like to private message with you about this in the future.
 
dutch029 said:
HPSAHBA works great with Ubuntu 18.04 and kernel 5.40. I can see all SAS and SATA drives. I have tried probably 20 different models. No fan problems ever. Booting directly to NVME using PCI adapter with SATA connection to the DVD drive. Remember to upgrade all firmware first to latest (P67 - 08/2015).

Tried compiling patches directly into PVE-Edge 5.12, but did not work. Compiling gave no errors, but I'm new to this so I moved on to using Debian Bullseye first before tackling PVE.

I'm just trying the same as you - Debian Bullseye on a DL 380 G7. I would suggest trying kernel 5.12 which has the patch for HPSAHBA. The other "patches" are symlinks to either 5.40 or 5.12 and as far as I can tell may not work. Upgrade the latest Debian from 5.10 to 5.12 and then install HPSAHBA.

Would like to private message with you about this in the future.
Click to expand...
Hi dutch029, my system is patched correctly and now it is ok. At the beginning I wrote this post to ask help to community. In the mean time I have found the solution and completed the guide. I hope that this guide will help you to make your server hbamode enabled or if you need help you can write to me.
 
emanuelebruno said:
Thank you for your reply. Does the activity led on drive bay continue to work correctly even with this third part controller raid?
Click to expand...
Yes, also the error LED flashes on boot. Haven't had a failed disk in this machine, so I cannot say that the fail led will also be on in case of a disk failure.
 
emanuelebruno said:
Hi dutch029, my system is patched correctly and now it is ok. At the beginning I wrote this post to ask help to community. In the mean time I have found the solution and completed the guide. I hope that this guide will help you to make your server hbamode enabled or if you need help you can write to me.
Click to expand...
Thanks for the guide. Very helpful!

Where did you flash the P410i? When I'm in the hpsahba directory and try "hpsahba -i /dev/sg1" it says command not found. I flashed a bunch of cards in Ubuntu 18 into HBA mode. Did you do that in Proxmox? Not sure what the difference is.
 
dutch029 said:
Thanks for the guide. Very helpful!

Where did you flash the P410i? When I'm in the hpsahba directory and try "hpsahba -i /dev/sg1" it says command not found. I flashed a bunch of cards in Ubuntu 18 into HBA mode. Did you do that in Proxmox? Not sure what the difference is.
Click to expand...
No flash is needed if your P410i controller firmware is 6.64.
To launch the hpsahba command and if you have followed my guide, you have to digit the following command:

cd ~/hpsahba/
./hpsahba -i /dev/sg1

Have a nice day.
 
Hi,
I get the following error:

root@stiga:~/hpsahba/contrib/dkms# dkms install --force hpsa-dkms/1.0

Kernel preparation unnecessary for this kernel. Skipping...

Building module:
cleaning build area...
make -j80 KERNELRELEASE=5.13.19-4-pve....(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.13.19-4-pve (x86_64)
Consult /var/lib/dkms/hpsa-dkms/1.0/build/make.log for more information.
root@stiga:~/hpsahba/contrib/dkms#

/var/lib/dkms/hpsa-dkms/1.0/build/make.log :

Here is the log content:

DKMS make.log for hpsa-dkms-1.0 for kernel 5.13.19-4-pve (x86_64)
Sun 06 Mar 2022 12:49:51 AM +03
make -C /lib/modules/5.13.19-4-pve/build M=/var/lib/dkms/hpsa-dkms/1.0/build modules
make[1]: Entering directory '/usr/src/linux-headers-5.13.19-4-pve'
CC [M] /var/lib/dkms/hpsa-dkms/1.0/build/hpsa.o
/var/lib/dkms/hpsa-dkms/1.0/build/hpsa.c:82:25: error: expected declaration specifiers or ‘...’ before string constant
82 | MODULE_SUPPORTED_DEVICE("HP Smart Array Controllers");
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [scripts/Makefile.build:281: /var/lib/dkms/hpsa-dkms/1.0/build/hpsa.o] Error 1
make[1]: *** [Makefile:1879: /var/lib/dkms/hpsa-dkms/1.0/build] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.13.19-4-pve'
make: *** [Makefile:15: default] Error 2
 
alpali2020 said:
Hi,
I get the following error:

root@stiga:~/hpsahba/contrib/dkms# dkms install --force hpsa-dkms/1.0

Kernel preparation unnecessary for this kernel. Skipping...

Building module:
cleaning build area...
make -j80 KERNELRELEASE=5.13.19-4-pve....(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.13.19-4-pve (x86_64)
Consult /var/lib/dkms/hpsa-dkms/1.0/build/make.log for more information.
root@stiga:~/hpsahba/contrib/dkms#

/var/lib/dkms/hpsa-dkms/1.0/build/make.log :

[...]
Click to expand...
Hi, did you try to delete and re-create dkms symbolic link?

cd ~/hpsahba/contrib/dkms rm /var/lib/dkms/hpsa-dkms/ -R dkms add ./ dkms install --force hpsa-dkms/1.0
...and so on..

Have a nice day.
 
alpali2020 said:
Hi,
I get the following error:

root@stiga:~/hpsahba/contrib/dkms# dkms install --force hpsa-dkms/1.0

Kernel preparation unnecessary for this kernel. Skipping...

Building module:
cleaning build area...
make -j80 KERNELRELEASE=5.13.19-4-pve....(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.13.19-4-pve (x86_64)
Consult /var/lib/dkms/hpsa-dkms/1.0/build/make.log for more information.
root@stiga:~/hpsahba/contrib/dkms#

/var/lib/dkms/hpsa-dkms/1.0/build/make.log :

Here is the log content:

DKMS make.log for hpsa-dkms-1.0 for kernel 5.13.19-4-pve (x86_64)
Sun 06 Mar 2022 12:49:51 AM +03
make -C /lib/modules/5.13.19-4-pve/build M=/var/lib/dkms/hpsa-dkms/1.0/build modules
make[1]: Entering directory '/usr/src/linux-headers-5.13.19-4-pve'
CC [M] /var/lib/dkms/hpsa-dkms/1.0/build/hpsa.o
/var/lib/dkms/hpsa-dkms/1.0/build/hpsa.c:82:25: error: expected declaration specifiers or ‘...’ before string constant
82 | MODULE_SUPPORTED_DEVICE("HP Smart Array Controllers");
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [scripts/Makefile.build:281: /var/lib/dkms/hpsa-dkms/1.0/build/hpsa.o] Error 1
make[1]: *** [Makefile:1879: /var/lib/dkms/hpsa-dkms/1.0/build] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.13.19-4-pve'
make: *** [Makefile:15: default] Error 2
Click to expand...
Hello,

for me this does not work with kernel 5.15. It's the same error message, re-create dkms symbolic link also didn't help.
Can someone help?
 
emanuelebruno said:
Hi uiffu,
did you edit Makefile and patch.sh according to your needs as in my guide above ?

View attachment 37821 View attachment 37822
Click to expand...
Hello,
yes, I checked the folder from Makefile and added the code to path.sh.
But I ran dkms install --force hpsa-dkms/1.0 with parameter for kernel 5.15.xx, because pve was running with 5.11 at this point. Is it mandatory to have the new kernel bootet for this to work?
Thanks
 
uiffiu said:
Hello,

for me this does not work with kernel 5.15. It's the same error message, re-create dkms symbolic link also didn't help.
Can someone help?
Click to expand...
I had a similar issue I am now rebooting right before doing the git for hpsahba. that way when VERSION gets set it is set on the running and the latest rev.
 
Finally got this working.. Can't tell you how much hassle this was. It's my penalty for buying old stuff I guess. A couple of notes on your instructions.

You should include a reboot before doing the "git clone" this sets the version correctly after the upgrade.

One of the wgets in patch.sh is copied wrong, not a bg deal but for the copy&paste moneys (like myself) it's confusing

Also the line:
echo "options hpsa hpsa_use_nvram_hba_flag=1" > /etc/modprobe.d/hpsa.co
should read:
echo "options hpsa hpsa_use_nvram_hba_flag=1" > /etc/modprobe.d/hpsa.conf


for some reason after booting I always have to:

modprobe -r hpsa
modprobe hpsa

Any idea why this loads wrong?

My hpsa.conf is as follows:

rick@pve02:~$ cat /etc/modprobe.d/hpsa.conf
options hpsa hpsa_use_nvram_hba_flag=1


Sensation instructions though, really appreciate it
 
Last edited:
  • Like
Reactions: prototype
I maintain a fork of the hpsahba repo that is proxmox-friendly by default. It addresses issues like the hardcoded kernel version in the upstream version.
It contains an install script for the dkms module which does all the work of installing the kernel module for you.

Today I've implemented even more changes which aim to fix all of the open issues on upstream:
  • hpsa_use_nvram_hba_flag=1 is now set by default, without the need to modprobe
  • added a (possible) fix for the dkms autoinstall hook failing after every kernel update.
  • incremented the dkms module version to prevent some issues
You can find the github repo here.
 
  • Like
Reactions: wbk and LnxBil
Rick1 said:
for some reason after booting I always have to:

modprobe -r hpsa
modprobe hpsa

Any idea why this loads wrong?
Click to expand...
I've also had problems with that. IIRC it had something to do with the system for some reason loading the stock version of hpsa, then failing to set the hba_flag.

In the past I had a systemd unit which would do the modprobe before zfs attempts to scan the drives, as the modprobe conf sometimes would not run correctly.

Since I've enabled hpsa_use_nvram_hba_flag=1 by default in the kernel patches, it works out of the box without setting the option manually
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back