[TUTORIAL] Configuring Fusion-Io (SanDisk) ioDrive, ioDrive2, ioScale and ioScale2 cards with Proxmox

[mrwizardno2]

Heads up: Had some trouble building DKMS with the PVE 8.1 release and new kernel. Ended up manually building and loading the module with insmod to get my lab back up.

 

[macgro1]

it's been my case... as well, rectified:
git clone https://github.com/RemixVSL/iomemory-vsl
cd iomemory-vsl/
sudo make dkms

 

[MJFlanigan]

How did you get it to run on proxmox 8?
What kernel are you using?

I am currently on proxmox 7.4-17/513c62be (running kernel: 6.1.15-1-pve)

but i am afraid to upgrade ...
Installing them was a nightmare... and i don't want to start all that again...

I would like to know the answer to this as well. I just bought a card from dell and I would like to use it in my proxmox server but I have no idea based on these instructions how to install this thing at all. I read though the repo and it's still unclear how to do this on proxmox. Is there a new guide for 8 out yet?

 

[Jan Panoch]

hello,

here is a updated guide for Proxmox v8.x (successfully build on proxmox 8.1.4 with kernel 6.5.11-7-pve):

apt update && apt --assume-yes install git zip unzip pve-headers proxmox-headers-`uname -r` && apt --assume-yes upgrade && apt --assume-yes autoremove && \
mkdir /home/temp  && cd /home/temp  && \
git clone https://github.com/RemixVSL/iomemory-vsl && \
wget -O fio-common_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/pd2ohfaufhwqc34/fio-common_3.2.16.1731-1.0_amd64.deb?dl=1 && \
wget -O fio-firmware-fusion_3.2.16.20180821-1_all.deb https://www.dropbox.com/s/kcn5agi6lyikicf/fio-firmware-fusion_3.2.16.20180821-1_all.deb?dl=1 && \
wget -O fio-sysvinit_3.2.16.1731-1.0_all.deb https://www.dropbox.com/s/g39l6lg9of6eqze/fio-sysvinit_3.2.16.1731-1.0_all.deb?dl=1 && \
wget -O fio-util_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/57huby17mteg6wp/fio-util_3.2.16.1731-1.0_amd64.deb?dl=1 && \
apt update && apt --assume-yes install gcc fakeroot build-essential debhelper rsync dkms && \
cp -r /home/temp/iomemory-vsl/root/usr/src/iomemory-vsl-3.2.16 /usr/src/ && \
mkdir -p /var/lib/dkms/iomemory-vsl/3.2.16/build && \
ln -s /usr/src/iomemory-vsl-3.2.16 /var/lib/dkms/iomemory-vsl/3.2.16/source && \
dkms build -m iomemory-vsl -v 3.2.16 && \
dkms install -m iomemory-vsl -v 3.2.16 && \
modprobe iomemory-vsl && \
cd /home/temp && \
dpkg -i fio-firmware-fusion_3.2.16.20180821-1_all.deb fio-util_3.2.16.1731-1.0_amd64.deb fio-sysvinit_3.2.16.1731-1.0_all.deb fio-common_3.2.16.1731-1.0_amd64.deb

btw. if you stayed - like me - on older kernel 5.15 with proxmox 8, because iomemory-vsl cannot be compiled with newer one, you have to install proper headers and compile driver a bit different way:

apt install proxmox-headers-6.5.11-7-pve
dkms build -m iomemory-vsl -v 3.2.16 -k 6.5.11-7-pve
dkms install -m iomemory-vsl -v 3.2.16 -k 6.5.11-7-pve

There are just minor changes from 7.x guide, it looks like the real work was done by RemixVSL who updated driver sources to support newer kernels..

Jan

 

[Jan Panoch]

I would like to know the answer to this as well. I just bought a card from dell and I would like to use it in my proxmox server but I have no idea based on these instructions how to install this thing at all. I read though the repo and it's still unclear how to do this on proxmox. Is there a new guide for 8 out yet?

I just posted updated guide for Proxmox 8.x

 

[Jan Panoch]

hmm, proxmox kernel proxmox-kernel-6.8.4-2-pve-signed has some changes, need to update driver sources from github repo.
i prepare small script to update:

#!/bin/sh
cd /home/temp
rm -Rf iomemory-vsl
git clone https://github.com/RemixVSL/iomemory-vsl
cp -r /home/temp/iomemory-vsl/root/usr/src/iomemory-vsl-3.2.16 /usr/src/

 

[Jan Panoch]

EDIT: fixed - i forgot to add iommu=pt to boot parameters.

There is a problem with kernel 6.8.x

snippet from dmesg

[    8.227469] <6>fioinf Copyright (c) 2006-2014 Fusion-io, Inc. (acquired by SanDisk Corp. 2014)
[   13.058621] <6>fioinf ioDrive 0000:08:00.0: Found device fct0 (Fusion-io ioScale 3.20TB 0000:08:00.0) on pipeline 0
[   13.059061] <3>fioerr Fusion-io ioScale 3.20TB 0000:08:00.0: failed to map append request
[   13.059063] <3>fioerr Fusion-io ioScale 3.20TB 0000:08:00.0: request page program 000000007bca9ea9 failed -22
[   18.862619] <6>fioinf ioDrive 0000:08:00.0: Found device fct0 (Fusion-io ioScale 3.20TB 0000:08:00.0) on pipeline 0
[   18.862976] <6>fioinf Fusion-io ioScale 3.20TB 0000:08:00.0: probed fct0
[   18.862979] <6>fioinf Fusion-io ioScale 3.20TB 0000:08:00.0: Attaching explicitly disabled
[   18.862987] <3>fioerr Fusion-io ioScale 3.20TB 0000:08:00.0: auto attach failed with error EINVAL: Invalid argument
[   35.664271] <6>fioinf Fusion-io ioScale 3.20TB 0000:08:00.0: IOCTL 0xc030684d unsupported in minimal mode
[   35.664419] <6>fioinf Fusion-io ioScale 3.20TB 0000:08:00.0: IOCTL 0x80286828 unsupported in minimal mode

fio-status -a

Found 1 ioMemory device in this system
Driver version: 3.2.16 build 1731

Adapter: Single Controller Adapter
    Fusion-io ioScale 3.20TB, Product Number:F11-002-3T20-CS-0001, SN:1439D178B, FIO SN:1439D178B
    ioDrive2 Adapter Controller, PN:PA005064001
    External Power: NOT connected
    PCIe Power limit threshold: 24.75W
    PCIe slot available power: unavailable
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Connected ioMemory modules:
      fct0:    Product Number:F11-002-3T20-CS-0001, SN:1439D178B

fct0    Status unknown: Driver is in MINIMAL MODE:
        Device has a hardware failure
    ioDrive2 Adapter Controller, Product Number:F11-002-3T20-CS-0001, SN:1439D178B
!! --->    There are active errors or warnings on this device!  Read below for details.
    ioDrive2 Adapter Controller, PN:PA005064001
    SMP(AVR) Versions: App Version: 1.0.21.0, Boot Version: 1.0.6.1
    Located in slot 0 Center of ioDrive2 Adapter Controller SN:1439D178B
    Powerloss protection: not available
    PCI:08:00.0, Slot Number:1
    Vendor:1aed, Device:2001, Sub vendor:1aed, Sub device:2001
    Firmware v7.1.17, rev 116786 Public
    Geometry and capacity information not available.
    Format: not low-level formatted
    PCIe slot available power: unavailable
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Internal temperature: 46.76 degC, max 46.76 degC
    Internal voltage: avg 1.02V, max 1.02V
    Aux voltage: avg 2.48V, max 2.48V
    Rated PBW: 20.00 PB
    Lifetime data volumes:
       Physical bytes written: 0
       Physical bytes read   : 0
    RAM usage:
       Current: 0 bytes
       Peak   : 0 bytes

    ACTIVE WARNINGS:
        The ioMemory is currently running in a minimal state.

kernel 6.5.x is working normally (tested on 6.5.13-5-pve)

i submitted a issue on RemixVSL github https://github.com/RemixVSL/iomemory-vsl/issues/129