PCIeX SSD doesn't work in Proxmox v3.1 [OCZ RevoDrive3]

X

XATRIX

Active Member
Feb 11, 2013
13
0
41
38
Ukraine/Kiev
Hi, i have troubles with my newly purchased SSD device OCZ RevoDrive 3 (RVD3-FHPX4-120G). I've installed it into the PCIeX raiser card slot, and booted my server up. As long as i can see, the device is listed in #lspci output.

Code: 
root@ve1-ua:~# lspci | grep -i ocz 02:00.0 SCSI storage controller: OCZ Technology Group, Inc. Device 1021 (rev 02)

But i don't have any block device initialized. Completely nothing in dmesg, related to OCZ or SSD or sdX devices.
I've downloaded official divers from the http://ocz.com/enterprise/download/drivers (Z-Drive R4 Linux Drivers). The version i chose download, was Ubuntu 10.04 LTS 2.6.32 62.7KB, because it suites my kernel most. I've installed it using dpkg -i oczpcie-ubuntu10.04-v4.0.551.x86_64.deb.
Later on, i modprobbed oczpcie oczvca modules into the kernel, but still have no device initialized in. I tried to load/unload mvsas module as well. Still no go.

Also this article says kernel 2.6.32 has the definitions for RevoDrive disks: http://cateee.net/lkddb/web-lkddb/SCSI_MVSAS.html

But my proxmox hardware node doesn't even try to initialize the PCIeX device.

Code: 
root@ve1-ua:~# pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-24 (running version: 3.1-24/060bd5a6)
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-19-pve: 2.6.32-96
pve-kernel-2.6.32-16-pve: 2.6.32-82
pve-kernel-2.6.32-22-pve: 2.6.32-107
pve-kernel-2.6.32-17-pve: 2.6.32-83
pve-kernel-2.6.32-26-pve: 2.6.32-114
pve-kernel-2.6.32-18-pve: 2.6.32-88
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-2
pve-cluster: 3.0-8
qemu-server: 3.1-8
pve-firmware: 1.0-23
libpve-common-perl: 3.0-9
libpve-access-control: 3.0-8
libpve-storage-perl: 3.0-18
pve-libspice-server1: 0.12.4-2
vncterm: 1.1-6
vzctl: 4.0-1pve4
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.1-1

I tried to install Ubuntu-server LTS 12.04.3 and my device works there:
Code: 
root@vox2-clu-ua:~# uname -a
Linux vox2-clu-ua 3.8.0-34-generic #49~precise1-Ubuntu SMP Wed Nov 13 18:05:00 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Code: 
root@vox2-clu-ua:~# hdparm -I /dev/sdd

/dev/sdd:

ATA device, with non-removable media
        Model Number:       OCZ-REVODRIVE3                          
        Serial Number:      OCZ-OD3Q24W51H54MCVF
        Firmware Revision:  2.25    
        Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Used: unknown (minor revision code 0x0110) 
        Supported: 8 7 6 5 
        Likely used: 8
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  117231408
        LBA48  user addressable sectors:  117231408
        Logical  Sector size:                   512 bytes
        Physical Sector size:                   512 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:       57241 MBytes
        device size with M = 1000*1000:       60022 MBytes (60 GB)
        cache/buffer size  = unknown
        Nominal Media Rotation Rate: Solid State Device
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        Advanced power management level: 254
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4 
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
                Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
           *    Advanced Power Management feature set
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
           *    48-bit Address feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    WRITE_{DMA|MULTIPLE}_FUA_EXT
           *    64-bit World wide name
           *    IDLE_IMMEDIATE with UNLOAD
                Write-Read-Verify feature set
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Host-initiated interface power management
           *    Phy event counters
           *    unknown 76[14]
           *    unknown 76[15]
                DMA Setup Auto-Activate optimization
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Data Tables (AC5)
           *    SET MAX SETPASSWORD/UNLOCK DMA commands
           *    Data Set Management TRIM supported (limit 1 block)
           *    Deterministic read data after TRIM
Security: 
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
        not     supported: enhanced erase
        2min for SECURITY ERASE UNIT. 
Logical Unit WWN Device Identifier: 5e83a976b45d4d03
        NAA             : 5
        IEEE OUI        : e83a97
        Unique ID       : 6b45d4d03
Checksum: correct

Code: 
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.144328] scsi0 : mvsas
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152940] sas: phy-0:2 added to port-0:0, phy_mask:0x4 ( 200000000000000)
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152956] sas: phy-0:3 added to port-0:1, phy_mask:0x8 ( 300000000000000)
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152973] sas: DOING DISCOVERY on port 0, pid:253
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152975] sas: DONE DISCOVERY on port 0, pid:253, result:0
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152979] sas: DOING DISCOVERY on port 1, pid:253
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152991] sas: DONE DISCOVERY on port 1, pid:253, result:0
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.152996] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.153001] sas: ata7: end_device-0:0: dev error handler
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.319865] ata7.00: ATA-8: OCZ-REVODRIVE3, 2.25, max UDMA/133
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.328236] ata7.00: 117231408 sectors, multi 16: LBA48 NCQ (depth 31/32)
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.339861] ata7.00: configured for UDMA/133
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.348378] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.364553] scsi 0:0:0:0: Direct-Access     ATA      OCZ-REVODRIVE3   2.25 PQ: 0 ANSI: 5
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.373342] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.373387] sas: ata7: end_device-0:0: dev error handler
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.373418] sas: ata8: end_device-0:1: dev error handler
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.539942] ata8.00: ATA-8: OCZ-REVODRIVE3, 2.25, max UDMA/133
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.548693] ata8.00: 117231408 sectors, multi 16: LBA48 NCQ (depth 31/32)
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.559941] ata8.00: configured for UDMA/133
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.568661] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.584451] scsi 0:0:1:0: Direct-Access     ATA      OCZ-REVODRIVE3   2.25 PQ: 0 ANSI: 5
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593382] sd 0:0:0:0: [sdd] 117231408 512-byte logical blocks: (60.0 GB/55.8 GiB)
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593400] sd 0:0:0:0: Attached scsi generic sg3 type 0
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593631] sd 0:0:1:0: [sde] 117231408 512-byte logical blocks: (60.0 GB/55.8 GiB)
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593653] sd 0:0:1:0: Attached scsi generic sg4 type 0
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593691] sd 0:0:1:0: [sde] Write Protect is off
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593692] sd 0:0:1:0: [sde] Mode Sense: 00 3a 00 00
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.593698] sd 0:0:1:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.594423]  sde: unknown partition table
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.594518] sd 0:0:1:0: [sde] Attached SCSI disk
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.663613] sd 0:0:0:0: [sdd] Write Protect is off
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.672274] sd 0:0:0:0: [sdd] Mode Sense: 00 3a 00 00
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.672292] sd 0:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.683178]  sdd: unknown partition table
Dec 19 14:59:31 vox2-clu-ua kernel: [    8.692024] sd 0:0:0:0: [sdd] Attached SCSI disk

Code: 
root@vox2-clu-ua:~# modinfo mvsas
filename:       /lib/modules/3.8.0-34-generic/kernel/drivers/scsi/mvsas/mvsas.ko
license:        GPL
version:        0.8.16
description:    Marvell 88SE6440 SAS/SATA controller driver
author:         Jeff Garzik <jgarzik@pobox.com>
srcversion:     8C2F663CC279D2FBBDE5EDD
alias:          pci:v00001B85d00001084sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001083sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001080sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001044sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001043sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001042sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001041sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001040sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001022sv*sd*bc*sc*i*
alias:          pci:v00001B85d00001021sv*sd*bc*sc*i*
alias:          pci:v00001B4Bd00009485sv*sd00009480bc*sc*i*
alias:          pci:v00001B4Bd00009445sv*sd00009480bc*sc*i*
alias:          pci:v00001B4Bd00009480sv*sd00009480bc*sc*i*
alias:          pci:v00001103d00002760sv*sd*bc*sc*i*
alias:          pci:v00001103d00002744sv*sd*bc*sc*i*
alias:          pci:v00001103d00002740sv*sd*bc*sc*i*
alias:          pci:v00001103d00002722sv*sd*bc*sc*i*
alias:          pci:v00001103d00002721sv*sd*bc*sc*i*
alias:          pci:v00001103d00002720sv*sd*bc*sc*i*
alias:          pci:v00001103d00002710sv*sd*bc*sc*i*
alias:          pci:v00009005d00000450sv*sd*bc*sc*i*
alias:          pci:v000017D3d00001320sv*sd*bc*sc*i*
alias:          pci:v000017D3d00001300sv*sd*bc*sc*i*
alias:          pci:v000011ABd00009180sv*sd*bc*sc*i*
alias:          pci:v000011ABd00009480sv*sd*bc*sc*i*
alias:          pci:v000011ABd00006485sv*sd*bc*sc*i*
alias:          pci:v000011ABd00006440sv*sd*bc*sc*i*
alias:          pci:v000011ABd00006440sv*sd00006480bc*sc*i*
alias:          pci:v000011ABd00006340sv*sd*bc*sc*i*
alias:          pci:v000011ABd00006320sv*sd*bc*sc*i*
depends:        libsas,scsi_transport_sas
intree:         Y
vermagic:       3.8.0-34-generic SMP mod_unload modversions 
parm:           collector:
        If greater than one, tells the SAS Layer to run in Task Collector
        Mode.  If 1 or 0, tells the SAS Layer to run in Direct Mode.
        The mvsas SAS LLDD supports both modes.
        Default: 1 (Direct Mode).
 (int)
 
Hi. There's no driver for i think. The only one driver i can find is "mvsas" version 0.8.16, which works like a charm, with full speed. But the proxmox kernel has 0.8.2 or so. And doesn't even try to detect the device (as i can see in dmesg). In Ubuntu LTS 12 / 13.xx - the device works pretty well.

P.S And how am i suppose to install .rpm RedHat 6.4 driver, using debian apt-get ?
 
XATRIX said:
Hi. There's no driver for i think. The only one driver i can find is "mvsas" version 0.8.16, which works like a charm, with full speed. But the proxmox kernel has 0.8.2 or so. And doesn't even try to detect the device (as i can see in dmesg). In Ubuntu LTS 12 / 13.xx - the device works pretty well.

P.S And how am i suppose to install .rpm RedHat 6.4 driver, using debian apt-get ?
Click to expand...

from http://ocz.com/enterprise/download/drivers:

http://ocz.com/enterprise/download/drivers/oczpcie-centos_rhel_oracle6.x-v4.0.551.x86_64.rpm

you can use "alien" to install rpm. (but maybe it's better to decompress it before to see what's inside exactly)
 
Seems like there's still no go...
root@ve1-ua:~# modprobe oczpcie
ERROR: could not insert 'oczpcie': Exec format error
root@ve1-ua:~# modprobe oczvca
ERROR: could not insert 'oczvca': Exec format error

What about the update to mvsas module ?
 
Try something like:
modprobe mvsas
echo 1b85 1021 > /sys/bus/pci/drivers/mvsas/new_id
Click to expand...
If it not helps, then you can try backporting newer mvsas module or wait until RHEL7 OpenVZ kernel is out and adopted to PVE.
 
That's what i have:
mvsas 0000:02:00.0: mvsas: driver version 0.8.2
mvsas 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
mvsas 0000:02:00.0: setting latency timer to 64
mvsas 0000:02:00.0: PCI INT A disabled
mvsas: probe of 0000:02:00.0 failed with error -12

Can you advice how much time do i have to wait until RHEL7 released ?
And i can't find the sources of the driver in web ,to compile it myself :(
 
Currently, only first beta of RHEL7 was released. I think it should be released in few months. Have no idea how long it will take to release OpenVZ version of its kernel.
You can find fresh driver in newer versions of kernel source, but I don't think that it can be easily backported. But you can try :)
 
Udo, can i still use openvz container virtualization in case of using 3.10 kernel ? Or i have to stick to KVM in this case ?
 
That's pretty bad :( But is it possible to path 3.10 kernel for OpenVZ ? Ok, if there's no way out in my case, i see no further way, but to install debian 7, and later on, upgrade to proxmox when RHEL7 will be released...
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back