[TUTORIAL] Configuring Fusion-Io (SanDisk) SX300, SX350 and PX600 cards with Proxmox

[Vladimir Bulgaru]

!!! Disclaimer !!!
This guide is yet to be tested. Looking for an owner of SX300, SX350 or PX600 cards to test this on a live platform and smooth out all the inconsistencies.

The problems

1. Fusion-Io drivers were designed with Enterprise OSs in mind (RHEL, SLES, ESXi, etc). Later on Sandisk was kind enough to expand the drivers to other OSs (updated list is here: https://link-app.sandisk.com/Home/SoftwareDownload). But you cannot get these drivers to work directly on Proxmox and you have to recompile them from source for your kernel.
2. Moreover, the kernels do get updated and it would be pain to lose the drive with every update.

The solution

1. Download all the files. Here's the cheatsheet, so you don't have to waste plenty of time on this (it creates a folder /home/temp where all the files are downloaded automatically:
   

   mkdir /home/temp && cd /home/temp && \
   wget -O iomemory-vsl4_4.3.5.1121-1.0.tar.gz https://www.dropbox.com/s/qqzj0i5ysye8ipu/iomemory-vsl4_4.3.5.1121-1.0.tar.gz?dl=1 && \
   wget -O fio-firmware-fusion_4.3.5.20190313-1_all.deb https://www.dropbox.com/s/vcud8bh1pcsiu35/fio-firmware-fusion_4.3.5.20190313-1_all.deb?dl=1 && \
   wget -O fio-sysvinit_4.3.5.1121-1.0_all.deb https://www.dropbox.com/s/vrhnam9r6b77xwc/fio-sysvinit_4.3.5.1121-1.0_all.deb?dl=1 && \
   wget -O fio-util_4.3.5.1121-1.0_amd64.deb https://www.dropbox.com/s/zbj8rwnur79j233/fio-util_4.3.5.1121-1.0_amd64.deb?dl=1 && \
   tar zxvf iomemory-vsl4_4.3.5.1121-1.0.tar.gz && cd iomemory-vsl4_4.3.5.1121

2. Get all the tools you need in order to compile the drivers from source (please be aware that at the time of reading the kernel is likely to be newer):

   apt-get install gcc fakeroot build-essential debhelper rsync pve-headers dkms

3. Create the DKMS config file and edit it via the command:

   nano /home/temp/iomemory-vsl4_4.3.5.1121/root/usr/src/iomemory-vsl-4.3.5/dkms.conf

   The content of the DKMS will be:

   MAKE="'make' DKMS_KERNEL_VERSION=$kernelver"
   CLEAN="'make' clean"
   BUILT_MODULE_NAME=iomemory-vsl
   BUILT_MODULE_LOCATION=''
   PACKAGE_NAME=iomemory-vsl
   PACKAGE_VERSION=4.3.5.1121
   DEST_MODULE_LOCATION=/kernel/drivers/block
   AUTOINSTALL="Yes"
   REMAKE_INITRD=yes

4. Prepare the source for being compiled, compile the drivers and load the drivers into the kernel:
   

   cp -r /home/temp/iomemory-vsl4_4.3.5.1121/root/usr/src/iomemory-vsl-4.3.5 /usr/src/ && \
   mkdir -p /var/lib/dkms/iomemory-vsl/4.3.5/build && \
   ln -s /usr/src/iomemory-vsl-4.3.5 /var/lib/dkms/iomemory-vsl/4.3.5/source && \
   dkms build -m iomemory-vsl -v 4.3.5 && \
   dkms install -m iomemory-vsl -v 4.3.5 && \
   modprobe iomemory-vsl

5. Install all utilities and firmware:
   

   cd /home/temp && \
   dpkg -i fio-firmware-fusion_4.3.5.20190313-1_all.deb fio-util_4.3.5.1121-1.0_amd64.deb fio-sysvinit_4.3.5.1121-1.0_all.deb

6. You may need to compile the drivers for the other kernels present on the system:
   

   ls /var/lib/initramfs-tools | sudo xargs -n1 /usr/lib/dkms/dkms_autoinstaller start

7. Reboot the OS and make sure the device is attached after the reboot by running

   fio-status -a

8. Format the device according to your needs and enjoy.

I'm still waiting to see how the device holds after the kernel update, just to make sure it's production ready. Other than that, seems to work properly. I've managed to create LVM-thin pool and import the VMs on it. The IO is very subpar (topic for another thread), but i was glad that a very decent hardware may still be used with the latest OSs.

Hope this helps a couple of souls out there. And you can buy me a beer if it saved you a couple of sleepless nights

 

[Khaitzu]

Vlad- Awesome that you were able to turn this around so quickly. Thanks you! I just tried this and it failed. Also, a couple of corrections:
Step 3 should be:

touch ./root/usr/src/iomemory-vsl4-4.3.5/dkms.conf && nano ./root/usr/src/iomemory-vsl4-4.3.5/dkms.conf

Step 4's copy statement should be:

cp -r /home/temp/iomemory-vsl4-4.3.5.1121/root/usr/src/iomemory-vsl4-4.3.5 /usr/src/

I've attached my make.log from this directory

/var/lib/dkms/iomemory-vsl/4.3.5/build/

 

[Tourman]

I was able to get the 4.3.7 drivers to compile under 4.15.18-27-pve, however I had to edit license.c to change MODULE_LICENSE to GPL instead of Proprietary, otherwise I can't get the source code to compile.