[SOLVED] Tape Label TASK ERROR: media read error - read failed - do_scsi_pt failed with err EINVAL: Invalid argument

Jan 4, 2022
16
2
3
20
Hi there,

I am currently trying to label the media of our tape library,
but I'm getting a Task Error

2022-01-04T19:08:13+01:00: TASK ERROR: media read error - read failed - do_scsi_pt failed with err EINVAL: Invalid argument

I think it is related to the hardware we use: The Overland NEO 400s which drive is attached to the backup-server running Proxmox Backup Server.

What can I do to find out more about the error?

I get this error for several other tasks I start that are related to tape media

Library Information
Serial Number MXA134Z055
Product NEOseries
Currently Installed Library Firmware G.83 / 3.10e
Bootcode Firmware Revision 0.80
Barcode Reader SE625
Library Mode Manual, Random
WWide Node Name 500143801603F830
Library Controller Version LCM 1.1

Drive Information 1 (LUN)
Vendor ID HP
Product ID Ultrium 5-SCSI
Serial Number HU1140JVN4
Firmware Revision Z58W
World Wide ID - Port A 500143801603F832
Physical Drive Slot Number 1
Element Address 1
Library LUN Hosted By Drive Yes
Data Compression Yes
Interface Type SAS
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
17,125
526
133
Austria
www.proxmox.com
Also, is the media empty or contains encrypted data? Please can you test with an new/empty tape?
 
Jan 4, 2022
16
2
3
20
Hi
the media do not contain encrypted data, I sadly got no new media at hand right now.

I checked to upgrade the drive firmware, but sadly hp only offers a .exe I search a way to get around this right now.
At first I try to get some non paywalled .exe to extract the firmware of. (easy) then I will come back.
 
Jan 4, 2022
16
2
3
20
I managed to install the hp_ltt software with some tweaks to their proprietary rpm RHEL/Linux package. The tool says that the firmware description cant be read, but also reports that there are no errors with the drive whatsoever.

Because library and drive where used before by another company, i would conclude that the hardware/software is running.

I managed to test with a fresh media and it still shows this problem.

Do I need to install custom drivers on the server in order to use some devices? I read that PBS uses its own drivers..

Is it a problem that the labels follow this enumeration:
1641389325191.png
 
Jan 4, 2022
16
2
3
20
Okey i managed to get my hands on some firmware version that are a bit more latest. currently trying the firmware upgrade manually over pg_write_puffer, hope this thing will survive this attempt
 
Jan 4, 2022
16
2
3
20
I am able to execute other things just fine with the drive.
For Example:
Bash:
pmt cartridge-memory
cleanly reads the tapes memory

Bash:
#pmt erase --drive neos-drive --fast=false
erases the tape as expected.

I checked this by checking the state of the tape library web interface. Drive Status says "erasing" for a couple of minutes.
So I think that at least the drive is functional.

So in conclusion of today: Drive and library firmware is up to date (and reflashed) basic tape operations via pmt are working so far.
I think that my problem has something to do with the labeling mechanism of Proxmox Backup Server.

Do you have any ideas for further troubleshooting?
 
Jan 4, 2022
16
2
3
20
I checked back for the pmt erase: ended like this:

Bash:
root@srv-backup-1:~# pmt erase --drive neos-drive --fast=false --device /dev/tape/by-id/scsi-HU1140JVN4-sg
using device /dev/tape/by-id/scsi-HU1140JVN4-sg
Error: erase failed - Illegal Request, Additional sense: Invalid command operation code

A look into syslog, tells not really more:
Bash:
root@srv-backup-1:~# tail -f /var/log/syslog
Jan  6 02:30:43 srv-backup-1 proxmox-backup-proxy[1024]: scsi command failed: transport error
Jan  6 02:31:09 srv-backup-1 proxmox-backup-proxy[1024]: scsi command failed: transport error
Jan  6 02:31:09 srv-backup-1 proxmox-backup-proxy[1024]: unable to get changer 'neos' status - error reading element status: read element status (B8h) failed: scsi command failed: transport error
Jan  6 02:31:48 srv-backup-1 proxmox-backup-proxy[1024]: scsi command failed: transport error
 
Last edited:

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,248
1,047
164
34
Vienna
mhm.. could you try to erase the tape with the 'mt' command instead of pmt ? (you may have to install the 'mt-st' package for that to be available)
 
Jan 4, 2022
16
2
3
20
Thanks for your suggestions, i will try it today after lunch.

I further digged into tape error codes and library event codes and found some errors suggesting a drive clean. which was not mentioned anywhere else on the library interface but I give it a go as well.

Code:
22.01.05 16:00:05.80  Event - Drive Warn or Crit Tape Alert flag
     Event Code: 0x84 - tape alert
     Element number: 0x01, 1
     Drive number: 0x01, 1
     Tape Alert Flag: 0x20, 32
22.01.05 15:54:45.66  Event - Drive Warn or Crit Tape Alert flag
     Event Code: 0x84 - tape alert
     Element number: 0x01, 1
     Drive number: 0x01, 1
     Tape Alert Flag: 0x22, 34
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,248
1,047
164
34
Vienna
so after cleaning the drive via the administration hw panel, i erased a tape with fast=false.
so it did successfully erase then?

can you load such a tape into the drive and post the output of
Code:
proxmox-tape status --output-format json-pretty
?
 
Jan 4, 2022
16
2
3
20
so it did successfully erase then?

can you load such a tape into the drive and post the output of
Code:
proxmox-tape status --output-format json-pretty
?
Ofcourse, here is the output with the erased tape loaded to the drive:
JSON:
{
  "alert-flags": "(empty)",
  "block-number": 0,
  "blocksize": 0,
  "buffer-mode": 1,
  "bytes-read": 239127756800,
  "bytes-written": 24059665448960,
  "compression": true,
  "density": "LTO5",
  "file-number": 0,
  "manufactured": 1339632000,
  "medium-passes": 922,
  "medium-wearout": 0.057625,
  "product": "Ultrium 5-SCSI",
  "revision": "Z58W",
  "vendor": "HP",
  "volume-mounts": 102
}
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,248
1,047
164
34
Vienna
ok, thanks looks good so far (aside from the tape already being older (though 922 passes should not be a problem)

can you do the labeling on the cli (maybe theres a bit more info) and post the output?
Code:
proxmox-tape label --label-text "LABEL"
?
 
Jan 4, 2022
16
2
3
20
Hi there, here is what i got so far:
Code:
root@srv-backup-1:~# proxmox-tape label --label-text "ABR401L5" --drive neos-drive --output-format=text --pool neos-pool
TASK ERROR: media read error - read failed - do_scsi_pt failed with err EINVAL: Invalid argument

I dug a bit deeper to find our more details again on that omnios Invalid Argument:
From /var/log/syslog:
Code:
Jan 12 18:35:51 srv-backup-1 proxmox-backup-proxy[1034]: Illegal Request, Additional sense: Invalid field in cdb

from within the /var/log/proxmox-backup/tasks/archive
Code:
UPID:srv-backup-1:0000040A:00000AA6:00000090:61DF109A:label-media:neos\x2ddrive:root@pam: 61DF109A media read error - read failed - do_scsi_pt failed with err EINVAL: Invalid argument

From /proc/scsi/debug/*:
Code:
root@srv-backup-1:/proc/scsi/sg# tail *
^[[3~==> allow_dio <==
0

==> debug <==
max_active_device=5  def_reserved_size=32768
 >>> device=sg3 7:0:0:1   em=0 sg_tablesize=60 excl=0 open_cnt=1
   FD(1): timeout=60000ms bufflen=32768 (res)sgat=1 low_dma=0
   cmd_q=1 f_packid=0 k_orphan=0 closed=0
     act: id=0 blen=65536 t_o/elap=300000/2328ms sgat=2 op=0xb8

==> def_reserved_size <==
32768

==> device_hdr <==
host    chan    id      lun   type      opens   qdepth  busy    online

==> devices <==
0 2     0       0       0     1 256     0       1
0 2     1       0       0     1 256     0       1
7 0     0       0       1     1 256     0       1
7 0     0       1       8     1 256     1       1
5 0     0       0       5     1 1       0       1

==> device_strs <==
DELL            PERC H710               3.13
DELL            PERC H710               3.13
HP              Ultrium 5-SCSI          Z58W
HP              MSL G3 Series           G.83
PLDS            DVD-ROM DS-8D3SH        HD51

==> version <==
30536   3.5.36 [20140603]

from /proc/scsi/scsi
Code:
root@srv-backup-1:/proc/scsi# cat scsi
Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: DELL     Model: PERC H710        Rev: 3.13
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi0 Channel: 02 Id: 01 Lun: 00
  Vendor: DELL     Model: PERC H710        Rev: 3.13
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi7 Channel: 00 Id: 00 Lun: 00
  Vendor: HP       Model: Ultrium 5-SCSI   Rev: Z58W
  Type:   Sequential-Access                ANSI  SCSI revision: 06
Host: scsi7 Channel: 00 Id: 00 Lun: 01
  Vendor: HP       Model: MSL G3 Series    Rev: G.83
  Type:   Medium Changer                   ANSI  SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 00
  Vendor: PLDS     Model: DVD-ROM DS-8D3SH Rev: HD51
  Type:   CD-ROM                           ANSI  SCSI revision: 05

I will check back to the NOC tomorrow to do some hardware maintenance on other servers, totally will check bios settings of that machine, maybe there is something to be done about this.

Best Regards


Markus
 
Jan 4, 2022
16
2
3
20
I just discovered a different driver for the lto-5 drive from hp on the overland FTP server. I flash a new firmware and hope that this will solve the problem for now.

(no big hopes, its uploaded 2014, still 2 years newer but meeh..)
 
Jan 4, 2022
16
2
3
20
So with the newest official firmware from the overland FTP still no success, same behavior for the drive

I tried once again, flashing manually without the web interface, but this firmware, potentially a bit newer gets rejected as well.

Any ideas for bios/hardware settings? Like is there something like a mode or setting in common with scsi?
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,248
1,047
164
34
Vienna
mhmm.. ok it seems during the read of the first block, something goes wrong...
to further debug this, can you do the following commands (and post the output)?

Code:
pmt rewind --drive DRIVENAME
proxmox-tape scan --drive DRIVENAME

(replace DRIVENAME with the configured one ofc)
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,248
1,047
164
34
Vienna
oh and please do a 'pmt format' before...
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!