Enterprise SSD Showing 0B in size

justinclift · Saturday at 06:40

As an example, this is from my main development pc at my house:

Bash:

# journalctl -b 0|grep -i mpt3sas
Jun 11 15:55:59 home1 kernel: mpt3sas version 43.100.00.00 loaded
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: 63 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (65755132 kB)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 4k
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: MSI-X vectors supported: 96
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0:  0 32 32
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: High IOPs queues : disabled
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix0: PCI-MSI-X enabled: IRQ 123
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix1: PCI-MSI-X enabled: IRQ 124
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix2: PCI-MSI-X enabled: IRQ 125
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix3: PCI-MSI-X enabled: IRQ 126
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix4: PCI-MSI-X enabled: IRQ 127
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix5: PCI-MSI-X enabled: IRQ 128
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix6: PCI-MSI-X enabled: IRQ 129
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix7: PCI-MSI-X enabled: IRQ 130
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix8: PCI-MSI-X enabled: IRQ 131
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix9: PCI-MSI-X enabled: IRQ 132
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix10: PCI-MSI-X enabled: IRQ 133
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix11: PCI-MSI-X enabled: IRQ 134
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix12: PCI-MSI-X enabled: IRQ 135
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix13: PCI-MSI-X enabled: IRQ 136
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix14: PCI-MSI-X enabled: IRQ 137
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix15: PCI-MSI-X enabled: IRQ 138
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix16: PCI-MSI-X enabled: IRQ 139
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix17: PCI-MSI-X enabled: IRQ 140
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix18: PCI-MSI-X enabled: IRQ 141
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix19: PCI-MSI-X enabled: IRQ 142
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix20: PCI-MSI-X enabled: IRQ 143
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix21: PCI-MSI-X enabled: IRQ 144
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix22: PCI-MSI-X enabled: IRQ 145
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix23: PCI-MSI-X enabled: IRQ 146
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix24: PCI-MSI-X enabled: IRQ 147
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix25: PCI-MSI-X enabled: IRQ 148
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix26: PCI-MSI-X enabled: IRQ 149
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix27: PCI-MSI-X enabled: IRQ 150
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix28: PCI-MSI-X enabled: IRQ 151
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix29: PCI-MSI-X enabled: IRQ 152
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix30: PCI-MSI-X enabled: IRQ 153
Jun 11 15:55:59 home1 kernel: mpt3sas0-msix31: PCI-MSI-X enabled: IRQ 154
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: iomem(0x00000000fc340000), mapped(0x0000000001dd60e5), size(65536)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: ioport(0x000000000000e000), size(256)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 4k
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: sending message unit reset !!
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: message unit reset: SUCCESS
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: scatter gather: sge_in_main_msg(1), sge_per_chain(7), sge_per_io(128), chains_per_io(19)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: request pool(0x00000000ce7ab4db) - dma(0xbea00000): depth(9960), frame_size(128), pool_size(1245 kB)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: sense pool(0x000000005a1bce78) - dma(0xbd200000): depth(9747), element_size(96), pool_size (913 kB)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: reply pool(0x00000000cd016225) - dma(0xbd000000): depth(10024), frame_size(128), pool_size(1253 kB)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: config page(0x0000000074327627) - dma(0xbcfee000): size(512)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: Allocated physical memory: size(31600 kB)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: Current Controller Queue Depth(9744),Max Controller Queue Depth(9856)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: Scatter Gather Elements per IO(128)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: _base_display_fwpkg_version: complete
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: overriding NVDATA EEDPTagMode setting
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: LSISAS3008: FWVersion(16.00.12.00), ChipRevision(0x02)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: sending port enable !!
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: hba_port entry: 00000000d582d591, port: 255 is added to hba_port list
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: host_add: handle(0x0001), sas_addr(0x5003005702193930), phys(8)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: handle(0x9) sas_address(0x5000c5005fcd6fc9) port_type(0x1)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: handle(0xa) sas_address(0x5000c5005fcf0275) port_type(0x1)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: handle(0xb) sas_address(0x5000c50067fe762d) port_type(0x1)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: handle(0xc) sas_address(0x5000c5005fd052ed) port_type(0x1)
Jun 11 15:55:59 home1 kernel: mpt3sas_cm0: port enable: SUCCESS

justinclift · Saturday at 06:42

Random question. Have you tried one of these failing drives in your friends system, with their HBA?

Just in case it's something to do with this model of HBA.

justinclift · Saturday at 06:45

minic119 said:
mpt2sas_cm1

Ahhh. Just noticed some of those messages are saying mpt2sas rather than mpt3sas.

We'll need to have the grep use both terms:

Bash:

# journalctl -b 0 | grep -i -e mpt2sas -e mpt3sas

minic119 · Saturday at 07:04

Code:

root@Saradomin:~# journalctl -b 0 | grep -i -e mpt2sas -e mpt3sas
Jun 14 13:29:41 Saradomin kernel: mpt3sas version 43.100.00.00 loaded
Jun 14 13:29:41 Saradomin kernel: mpt3sas 0000:02:00.0: can't disable ASPM; OS doesn't have ASPM control
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (263996304 kB)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 4k
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: MSI-X vectors supported: 1
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0:  0 1 1
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: High IOPs queues : disabled
Jun 14 13:29:41 Saradomin kernel: mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 69
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: iomem(0x00000000c7440000), mapped(0x0000000093f226dd), size(65536)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: ioport(0x0000000000006000), size(256)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: CurrentHostPageSize is 0: Setting default host page size to 4k
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: sending message unit reset !!
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: message unit reset: SUCCESS
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: scatter gather: sge_in_main_msg(1), sge_per_chain(9), sge_per_io(128), chains_per_io(15)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: request pool(0x0000000057e09994) - dma(0x120180000): depth(3492), frame_size(128), pool_size(436 kB)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: sense pool(0x00000000f59db16b) - dma(0x120380000): depth(3367), element_size(96), pool_size (315 kB)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: reply pool(0x0000000031eaa143) - dma(0x120400000): depth(3556), frame_size(128), pool_size(444 kB)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: config page(0x0000000092a6a075) - dma(0x12030b000): size(512)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: Allocated physical memory: size(7579 kB)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: Current Controller Queue Depth(3364),Max Controller Queue Depth(3432)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: Scatter Gather Elements per IO(128)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: overriding NVDATA EEDPTagMode setting
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: LSISAS2008: FWVersion(20.00.07.00), ChipRevision(0x03)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: sending port enable !!
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: hba_port entry: 00000000387f6ec0, port: 255 is added to hba_port list
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: host_add: handle(0x0001), sas_addr(0x5d4ae520972d0900), phys(8)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x5003048001627dbf), phys(38)
Jun 14 13:29:41 Saradomin kernel: mpt3sas 0000:03:00.0: can't disable ASPM; OS doesn't have ASPM control
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0xa) sas_address(0x5003048001627d8c) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0xb) sas_address(0x5003048001627d8d) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0xc) sas_address(0x5003048001627d8e) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0xd) sas_address(0x5003048001627d8f) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0xe) sas_address(0x5003048001627d9c) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0xf) sas_address(0x5003048001627d9d) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0x10) sas_address(0x5003048001627d9e) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0x11) sas_address(0x5003048001627d9f) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0x12) sas_address(0x5003048001627da0) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0x13) sas_address(0x5003048001627da1) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: handle(0x14) sas_address(0x5003048001627dbd) port_type(0x1)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (263996304 kB)
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm0: port enable: SUCCESS
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size to 4k
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: MSI-X vectors supported: 16
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1:  0 16 16
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: High IOPs queues : disabled
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix0: PCI-MSI-X enabled: IRQ 75
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix1: PCI-MSI-X enabled: IRQ 76
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix2: PCI-MSI-X enabled: IRQ 77
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix3: PCI-MSI-X enabled: IRQ 78
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix4: PCI-MSI-X enabled: IRQ 79
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix5: PCI-MSI-X enabled: IRQ 80
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix6: PCI-MSI-X enabled: IRQ 81
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix7: PCI-MSI-X enabled: IRQ 82
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix8: PCI-MSI-X enabled: IRQ 83
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix9: PCI-MSI-X enabled: IRQ 84
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix10: PCI-MSI-X enabled: IRQ 85
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix11: PCI-MSI-X enabled: IRQ 86
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix12: PCI-MSI-X enabled: IRQ 87
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix13: PCI-MSI-X enabled: IRQ 88
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix14: PCI-MSI-X enabled: IRQ 89
Jun 14 13:29:41 Saradomin kernel: mpt2sas1-msix15: PCI-MSI-X enabled: IRQ 90
Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: iomem(0x00000000c7240000), mapped(0x00000000186d418e), size(65536)

Edit: I have also pasted the second half of the output but it is currently waiting moderator approval.

minic119 · Saturday at 07:05

Code:

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: ioport(0x0000000000005000), size(256)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: CurrentHostPageSize is 0: Setting default host page size to 4k

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: sending message unit reset !!

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: message unit reset: SUCCESS

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: scatter gather: sge_in_main_msg(1), sge_per_chain(9), sge_per_io(128), chains_per_io(15)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: request pool(0x000000003fb53c19) - dma(0x121e00000): depth(10368), frame_size(128), pool_size(1296 kB)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: sense pool(0x000000002ad17c2a) - dma(0x124000000): depth(10107), element_size(96), pool_size (947 kB)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: reply pool(0x000000005fb109a7) - dma(0x122400000): depth(10432), frame_size(128), pool_size(1304 kB)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: config page(0x00000000e2c951b7) - dma(0x1239d1000): size(512)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: Allocated physical memory: size(25141 kB)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: Current Controller Queue Depth(10104),Max Controller Queue Depth(10240)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: Scatter Gather Elements per IO(128)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: overriding NVDATA EEDPTagMode setting

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: LSISAS2308: FWVersion(20.00.07.00), ChipRevision(0x05)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: Protocol=(Initiator,Target), Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: sending port enable !!

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: hba_port entry: 00000000fdf00d16, port: 255 is added to hba_port list

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: host_add: handle(0x0001), sas_addr(0x500605b009211f20), phys(8)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x50001555480c223f), phys(37)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0xa) sas_address(0x50001555480c2200) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0xb) sas_address(0x50001555480c2201) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0xc) sas_address(0x50001555480c2202) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0xd) sas_address(0x50001555480c2203) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0xe) sas_address(0x50001555480c2204) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0xf) sas_address(0x50001555480c2205) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x10) sas_address(0x50001555480c2206) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x11) sas_address(0x50001555480c2207) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x12) sas_address(0x50001555480c2208) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x13) sas_address(0x50001555480c2209) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x14) sas_address(0x50001555480c220a) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x15) sas_address(0x50001555480c220b) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x16) sas_address(0x50001555480c220c) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x17) sas_address(0x50001555480c220d) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x18) sas_address(0x50001555480c220e) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x19) sas_address(0x50001555480c220f) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: handle(0x1a) sas_address(0x50001555480c223e) port_type(0x1)

Jun 14 13:29:41 Saradomin kernel: mpt2sas_cm1: port enable: SUCCESS

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2200)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x000a), sas_addr(0x50001555480c2200)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(0)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2201)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x000b), sas_addr(0x50001555480c2201)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(1)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2202)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x000c), sas_addr(0x50001555480c2202)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(2)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2203)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x000d), sas_addr(0x50001555480c2203)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(3)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2204)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x000e), sas_addr(0x50001555480c2204)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(4)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2205)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x000f), sas_addr(0x50001555480c2205)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(5)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2206)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0010), sas_addr(0x50001555480c2206)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(6)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2207)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0011), sas_addr(0x50001555480c2207)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(7)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2208)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0012), sas_addr(0x50001555480c2208)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(8)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c2209)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0013), sas_addr(0x50001555480c2209)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(9)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c220a)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0014), sas_addr(0x50001555480c220a)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(10)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c220b)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0015), sas_addr(0x50001555480c220b)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(11)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c220c)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0016), sas_addr(0x50001555480c220c)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(12)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c220d)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0017), sas_addr(0x50001555480c220d)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(13)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c220e)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0018), sas_addr(0x50001555480c220e)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(14)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c220f)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x0019), sas_addr(0x50001555480c220f)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(15)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c223e)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: removing handle(0x001a), sas_addr(0x50001555480c223e)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: enclosure logical id(0x50001555480c223f), slot(36)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: mpt3sas_transport_port_remove: removed: sas_addr(0x50001555480c223f)

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: expander_remove: handle(0x0009), sas_addr(0x50001555480c223f), port:255

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: sending message unit reset !!

Jun 14 13:29:52 Saradomin kernel: mpt2sas_cm1: message unit reset: SUCCESS

Jun 14 14:16:32 Saradomin kernel: mpt2sas_cm0: handle(0x15) sas_address(0x500003964c88053a) port_type(0x1)

Jun 14 14:17:17 Saradomin kernel: mpt2sas_cm0: mpt3sas_transport_port_remove: removed: sas_addr(0x500003964c88053a)

Jun 14 14:17:17 Saradomin kernel: mpt2sas_cm0: removing handle(0x0015), sas_addr(0x500003964c88053a)

Jun 14 14:17:17 Saradomin kernel: mpt2sas_cm0: enclosure logical id(0x5003048001627dbf), slot(6)

Jun 14 14:18:37 Saradomin kernel: mpt2sas_cm0: handle(0x15) sas_address(0x500003964c88062e) port_type(0x1)

Jun 14 16:33:06 Saradomin kernel: mpt2sas_cm0: mpt3sas_transport_port_remove: removed: sas_addr(0x500003964c88062e)

Jun 14 16:33:06 Saradomin kernel: mpt2sas_cm0: removing handle(0x0015), sas_addr(0x500003964c88062e)

Jun 14 16:33:06 Saradomin kernel: mpt2sas_cm0: enclosure logical id(0x5003048001627dbf), slot(6)

Jun 14 16:33:26 Saradomin kernel: mpt2sas_cm0: handle(0x15) sas_address(0x500003964c88062e) port_type(0x1)

minic119 · Saturday at 07:06

justinclift said:
Random question. Have you tried one of these failing drives in your friends system, with their HBA?

Just in case it's something to do with this model of HBA.

I have not as his HBA card is in a chassis that has 3.5 inch drive bays on the front of his server and mine are 2.5 inch drives

justinclift · Saturday at 07:15

minic119 said:
I have not as his HBA card is in a chassis that has 3.5 inch drive bays on the front of his server and mine are 2.5 inch drives

Ahhh. If you're both up for some experimentation one day, then you can technically (carefully) get a 2.5" drive (minus carrier) to slot into the back of the 3.5" drive and hook up to the SAS+power connector in the back. It'll then work and doesn't damage anything. Just don't try moving the server while it's hooked up like that.

Getting it back out again afterwards can be a bit of a pain, but also isn't toooooo much of a hassle if you have some patience.

justinclift · Saturday at 07:25

Hmmm, looking around for similar problems, other things kind of similar are showing up from a while back. Example here:

https://forum.proxmox.com/threads/help-with-lsi-card.104211/

That particular one was fixed with a different kernel too (much earlier than current release).

I'm really suspecting something to do with the kernel and/or kernel module is acting up when it talks to your card, but I have no clear idea how to work around it. That doesn't mean you're out of options though.

Are you familiar with providing options to kernel modules for them to use at boot time?

Asking because the mpt2sas / mpt3sas kernel module has a bunch of options that can be set:

Bash:

# modinfo mpt2sas   <-- not sure why, but running this gives the output for mpt3sas rather than mpt2sas
[...stuff...]
parm:           logging_level: bits for enabling additional logging info (default=0)
parm:           max_sectors:max sectors, range 64 to 32767  default=32767 (ushort)
parm:           missing_delay: device missing delay , io missing delay (array of int)
parm:           max_lun: max lun, default=16895  (ullong)
parm:           hbas_to_enumerate: 0 - enumerates both SAS 2.0 & SAS 3.0 generation HBAs
                  1 - enumerates only SAS 2.0 generation HBAs
                  2 - enumerates only SAS 3.0 generation HBAs (default=0) (ushort)
parm:           diag_buffer_enable: post diag buffers (TRACE=1/SNAPSHOT=2/EXTENDED=4/default=0) (int)
parm:           disable_discovery: disable discovery  (int)
parm:           prot_mask: host protection capabilities mask, def=7  (int)
parm:           enable_sdev_max_qd:Enable sdev max qd as can_queue, def=disabled(0) (bool)
parm:           multipath_on_hba:Multipath support to add same target device
                as many times as it is visible to HBA from various paths
                (by default:
                         SAS 2.0 & SAS 3.0 HBA - This will be disabled,
                         SAS 3.5 HBA - This will be enabled) (int)
parm:           host_tagset_enable:Shared host tagset enable/disable Default: enable(1) (int)
parm:           max_queue_depth: max controller queue depth  (int)
parm:           max_sgl_entries: max sg entries  (int)
parm:           msix_disable: disable msix routed interrupts (default=0) (int)
parm:           smp_affinity_enable:SMP affinity feature enable/disable Default: enable(1) (int)
parm:           max_msix_vectors: max msix vectors (int)
parm:           irqpoll_weight:irq poll weight (default= one fourth of HBA queue depth) (int)
parm:           mpt3sas_fwfault_debug: enable detection of firmware fault and halt firmware - (default=0)
parm:           perf_mode:Performance mode (only for Aero/Sea Generation), options:
                0 - balanced: high iops mode is enabled &
                interrupt coalescing is enabled only on high iops queues,
                1 - iops: high iops mode is disabled &
                interrupt coalescing is enabled on all queues,
                2 - latency: high iops mode is disabled &
                interrupt coalescing is enabled on all queues with timeout value 0xA,
                default - default perf_mode is 'balanced' (int)
parm:           poll_queues:Number of queues to be use for io_uring poll mode.
                This parameter is effective only if host_tagset_enable=1. &
                when poll_queues are enabled then &
                perf_mode is set to latency mode. &
                 (int)

In your shoes, I'd probably start looking into those options, and seeing if changing stuff has any kind of effect.

justinclift · Saturday at 07:29

@alexskysilk Any ideas with this?

minic119 · Saturday at 07:34

justinclift said:

Hmmm, looking around for similar problems, other things kind of similar are showing up from a while back. Example here:

https://forum.proxmox.com/threads/help-with-lsi-card.104211/

That particular one was fixed with a different kernel too (much earlier than current release).

I'm really suspecting something to do with the kernel and/or kernel module is acting up when it talks to your card, but I have no clear idea how to work around it. That doesn't mean you're out of options though.

Are you familiar with providing options to kernel modules for them to use at boot time?

Asking because the mpt2sas / mpt3sas kernel module has a bunch of options that can be set:

Bash:

# modinfo mpt2sas   <-- not sure why, but running this gives the output for mpt3sas rather than mpt2sas
[...stuff...]
parm:           logging_level: bits for enabling additional logging info (default=0)
parm:           max_sectors:max sectors, range 64 to 32767  default=32767 (ushort)
parm:           missing_delay: device missing delay , io missing delay (array of int)
parm:           max_lun: max lun, default=16895  (ullong)
parm:           hbas_to_enumerate: 0 - enumerates both SAS 2.0 & SAS 3.0 generation HBAs
                  1 - enumerates only SAS 2.0 generation HBAs
                  2 - enumerates only SAS 3.0 generation HBAs (default=0) (ushort)
parm:           diag_buffer_enable: post diag buffers (TRACE=1/SNAPSHOT=2/EXTENDED=4/default=0) (int)
parm:           disable_discovery: disable discovery  (int)
parm:           prot_mask: host protection capabilities mask, def=7  (int)
parm:           enable_sdev_max_qd:Enable sdev max qd as can_queue, def=disabled(0) (bool)
parm:           multipath_on_hba:Multipath support to add same target device
                as many times as it is visible to HBA from various paths
                (by default:
                         SAS 2.0 & SAS 3.0 HBA - This will be disabled,
                         SAS 3.5 HBA - This will be enabled) (int)
parm:           host_tagset_enable:Shared host tagset enable/disable Default: enable(1) (int)
parm:           max_queue_depth: max controller queue depth  (int)
parm:           max_sgl_entries: max sg entries  (int)
parm:           msix_disable: disable msix routed interrupts (default=0) (int)
parm:           smp_affinity_enable:SMP affinity feature enable/disable Default: enable(1) (int)
parm:           max_msix_vectors: max msix vectors (int)
parm:           irqpoll_weight:irq poll weight (default= one fourth of HBA queue depth) (int)
parm:           mpt3sas_fwfault_debug: enable detection of firmware fault and halt firmware - (default=0)
parm:           perf_mode:Performance mode (only for Aero/Sea Generation), options:
                0 - balanced: high iops mode is enabled &
                interrupt coalescing is enabled only on high iops queues,
                1 - iops: high iops mode is disabled &
                interrupt coalescing is enabled on all queues,
                2 - latency: high iops mode is disabled &
                interrupt coalescing is enabled on all queues with timeout value 0xA,
                default - default perf_mode is 'balanced' (int)
parm:           poll_queues:Number of queues to be use for io_uring poll mode.
                This parameter is effective only if host_tagset_enable=1. &
                when poll_queues are enabled then &
                perf_mode is set to latency mode. &
                 (int)

In your shoes, I'd probably start looking into those options, and seeing if changing stuff has any kind of effect.

I am not familiar with messing with the kernel modules at all, i have never even attempted, usually everything works pretty well but when i have ran into things it has been things i could find in the forums to correct whatever i messed up or hadn't configured yet.

justinclift · Saturday at 07:45

No worries. It's not actually super hard or anything.

If you take a look in the /etc/modprobe.d/ directory you'll see a bunch of text files ending in .conf.

This is my current home pc system as an example:

Bash:

# ls -al /etc/modprobe.d/
total 46
drwxr-xr-x   2 root root  11 Jun  6 19:24 ./
drwxr-xr-x 157 root root 268 Jun 12 12:25 ../
-rw-r--r--   1 root root 155 Jun  4 17:58 amd64-microcode-blacklist.conf
-rw-r--r--   1 root root  30 Jun  6 19:24 disable-softdog-reboot.conf
-rw-r--r--   1 root root 127 Jul 12  2023 dkms.conf
lrwxrwxrwx   1 root root  53 Feb 15 11:33 nvidia-blacklists-nouveau.conf -> /etc/alternatives/glx--nvidia-blacklists-nouveau.conf
lrwxrwxrwx   1 root root  43 Feb 15 11:33 nvidia.conf -> /etc/alternatives/glx--nvidia-modprobe.conf
lrwxrwxrwx   1 root root  45 Feb 15 11:33 nvidia-options.conf -> /etc/alternatives/nvidia--nvidia-options.conf
-rw-r--r--   1 root root 173 May 14 21:26 pve-blacklist.conf
-rw-r--r--   1 root root  49 May  8 10:15 vmware-fuse.conf
-rw-r--r--   1 root root 270 May  8 10:03 zfs.conf

Taking a look at a random one there (for ZFS):

Bash:

# cat zfs.conf
options zfs zfs_arc_min=1073741824
options zfs zfs_arc_max=2147483648

It's pretty straight forward. The layout is (one per line) "options NAME_OF_MODULE NAME_OF_PARAMETER=VALUE".

So, you'd create a new file called say "hbatest.conf" and then throw in what seems like an interesting option to try out. eg (random option picked):

options mpt2sas logging_level=1

When you reboot, the kernel reads the .conf files in that directory and applies any options it finds to the modules as they're loaded.

So, I reckon do some experimentation and see if you can get the drives to act differently, hopefully in a positive direction.

justinclift · Saturday at 07:54

I'd probably also try disabling msix, just to see if that makes a difference:

Code:

options mpt2sas msix_disable=1
options mpt3sas msix_disable=1

alexskysilk · Saturday at 17:19

I think the key to this mystery lies is "all the sudden."

what happened immediately prior to that boot? firmware update? kernel update?

minic119 · Sunday at 04:31

alexskysilk said:
I think the key to this mystery lies is "all the sudden."

what happened immediately prior to that boot? firmware update? kernel update?

an adrupt power off then just normal boot after, as far as i know, no updates were done prior to the reboot. The power off was caused by power loss to the building.

minic119 · Sunday at 15:45

justinclift said:
I'd probably also try disabling msix, just to see if that makes a difference:

Code:

options mpt2sas msix_disable=1 options mpt3sas msix_disable=1

I have enabled both those options as well as the logging_level=1 for both mpt2sas and mpt3sas, where can i get those extra logs it is making? if 1 means more logs that is. Nothing has changed as far as those 2 drives or even any of my other drives in the machine and all drives in my JBOD that are attached to a separate hba card are showing up and operational and i havent noticed a change in any disks since adding these options and rebooting. Do i need to run anything to apply those updates or does it pull from that directory on each boot no matter what has been added?

justinclift · Sunday at 17:54

minic119 said:
can i get those extra logs it is making

My initial assumption is they'd be going into the kernel output, but it's not something I've personally had to do... so I'm not 100% sure.

You can keep an eye on the running kernel output by running this in a terminal on the host:

Bash:

# journalctl -b 0 -n 50 -f

The -b 0 means "show the logs for this current boot", the -n 50 means "start from the last 50 lines of output" and the -f means "run in the foreground, outputting new lines as they arrive".

If that turns out to not be the correct output place, then some online searching/Googling should turn up the correct thing.

minic119 said:
Do i need to run anything to apply those updates or does it pull from that directory on each boot no matter what has been added?

Hmmm, I didn't think you'd need to run anything in order for them to be applied and used.

That being said, it probably wouldn't hurt to run an update-initramfs -u -k all (then reboot), just in case.

We're hitting the edges of my personal knowledge now though, thus my calling out to @alexskysilk who seems to know this area of stuff better.

minic119 · 2024-06-17T16:19:12+0200

@alexskysilk would you have any other reccommendations for options on the mpt2sas and the mpt3sas? I can provide any logs you need.

minic119 · 2024-06-17T19:23:36+0200

Big apologies the drives i thought were showing up in windows actually are not, there was a usb drive hooked up to my machine in the server rack and it was the same size as the SSD's i thought were bad, after removing that external and trying the effected drives again they do not show up in windows either so i guess they are bad. Thank you for the help and again i apologize for the mixup. The proxmox community is so ready to lend a helping hand and I greatly appreciate that!

justinclift · 2024-06-17T20:28:30+0200

Heh Heh Heh

No worries, we all make silly mistakes every now and then.

At least the problem of "what the heck is going on?" got solved.

alexskysilk · 2024-06-17T20:37:08+0200

minic119 said:
after removing that external and trying the effected drives again they do not show up in windows either so i guess they are bad.

This would be really concerning to me. What hardware were these connected to? those power supplies should have protected the devices on the low voltage rails from any spike or adverse condition; I'd effectively rule out its use in any meaningful application.

Enterprise SSD Showing 0B in size

Active Member

Active Member

Active Member

Member

Member

Member

Active Member

Active Member

Active Member

Member

Active Member

Active Member

Distinguished Member

Member

Member

Active Member

Member

Member

Active Member

Distinguished Member