Proxmox Backup on the Tapes

[flowbergit]

HP MSL6480 and I have problem with PBS and backup LOGS show only this and waiting all the time but on the screen of the drives, drive shows that correct tape is inside.

2024-01-04T19:44:03+01:00: Starting tape backup job 'BACKUP350:KS:Naped1:BACKUP_KS'
2024-01-04T19:44:03+01:00: update media online status
2024-01-04T19:44:03+01:00: media set uuid: 6bb56fee-1974-4448-9777-cc0e3701c72f
2024-01-04T19:44:03+01:00: found 12 groups
2024-01-04T19:44:03+01:00: backup snapshot "ns/KS/vm/129/2024-01-04T13:59:50Z"
2024-01-04T19:44:09+01:00: allocated new writable media 'BOY020L6'
2024-01-04T19:44:09+01:00: trying to load media 'BOY020L6' into drive 'Naped1'
2024-01-04T19:54:52+01:00: tape open failed - open drive 'Naped1' (/dev/tape/by-id/scsi-8026DCC735-sg) failed - wait_until_ready failed - got timeout
2024-01-04T19:54:52+01:00: Please insert media 'BOY020L6' into changer 'TASMY-KS'

 

[flowbergit]

Previously backup works like a charm, but after last update PBS and MSL firmware doesn't work anymore :-(

 

[dcsapak]

hi,
can you post the output of

proxmox-backup-manager versions --verbose

?

does the problem occur always or only with that tape? did it work in the past?
what does

proxmox-tape changer status <changer-name>

say?

 

[flowbergit]

hi,
can you post the output of

proxmox-backup-manager versions --verbose

?

does the problem occur always or only with that tape? did it work in the past?
what does

proxmox-tape changer status <changer-name>

say?

 proxmox-backup-manager versions --verbose
proxmox-backup                     3.0.1        running kernel: 6.5.11-6-pve
proxmox-backup-server              3.1.2-1      running version: 3.1.2     
proxmox-kernel-helper              8.1.0                                   
proxmox-kernel-6.5                 6.5.11-7                                 
proxmox-kernel-6.5.11-7-pve-signed 6.5.11-7                                 
proxmox-kernel-6.5.11-6-pve-signed 6.5.11-6                                 
ifupdown2                          3.2.0-1+pmx7                             
libjs-extjs                        7.0.0-4                                 
proxmox-backup-docs                3.1.2-1                                 
proxmox-backup-client              3.1.2-1                                 
proxmox-mail-forward               0.2.2                                   
proxmox-mini-journalreader         1.4.0                                   
proxmox-offline-mirror-helper      0.6.3                                   
proxmox-widget-toolkit             4.1.3                                   
pve-xtermjs                        5.3.0-2                                 
smartmontools                      7.3-pve1                                 
zfsutils-linux                     2.2.2-pve1

root@backup350:~# proxmox-tape changer status TASMY
┌────────────┬──────────┬────────────┬─────────────┐
│ entry-kind │ entry-id │ label-text │ loaded-slot │
╞════════════╪══════════╪════════════╪═════════════╡
│ drive      │        0 │ BOY020L6   │           1 │
├────────────┼──────────┼────────────┼─────────────┤
│ drive      │        1 │            │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        1 │            │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        2 │ BOY021L6   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        3 │ BOY022L6   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        4 │ BOY023L6   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        5 │ CLNU00L1   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        6 │ CLNU01L1   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        7 │ CLNU02L1   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        8 │ CLNU03L1   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │        9 │ CLNU04L1   │             │
├────────────┼──────────┼────────────┼─────────────┤
│ slot       │       10 │ CLNU05L1   │             │
└────────────┴──────────┴────────────┴─────────────┘

 

[flowbergit]

Problem appears after upgrade PBS and in the same time (new firmware for MSL). In the past works great

 

[dcsapak]

what does

proxmox-tape status --drive Naped1

show?

 

[flowbergit]

root@backup350:~# proxmox-tape status --drive Naped1
┌─────────────┬─────────┐
│ Name        │ Value   │
╞═════════════╪═════════╡
│ blocksize   │ 0       │
├─────────────┼─────────┤
│ density     │ LTO6    │
├─────────────┼─────────┤
│ compression │ 1       │
├─────────────┼─────────┤
│ buffer-mode │ 1       │
├─────────────┼─────────┤
│ alert-flags │ (empty) │
└─────────────┴─────────┘
root@backup350:~#

 

[dcsapak]

this looks like it's empty, can you do

proxmox-tape status --drive Naped2

too?

 

[flowbergit]

root@backup350:~# proxmox-tape status --drive Naped2
┌────────────────┬──────────────────────────┐
│ Name           │ Value                    │
╞════════════════╪══════════════════════════╡
│ blocksize      │ 0                        │
├────────────────┼──────────────────────────┤
│ density        │ LTO6                     │
├────────────────┼──────────────────────────┤
│ compression    │ 1                        │
├────────────────┼──────────────────────────┤
│ buffer-mode    │ 1                        │
├────────────────┼──────────────────────────┤
│ alert-flags    │ (empty)                  │
├────────────────┼──────────────────────────┤
│ file-number    │ 0                        │
├────────────────┼──────────────────────────┤
│ block-number   │ 0                        │
├────────────────┼──────────────────────────┤
│ manufactured   │ Sat Dec  6 01:00:00 2014 │
├────────────────┼──────────────────────────┤
│ bytes-written  │ 28 MiB                   │
├────────────────┼──────────────────────────┤
│ bytes-read     │ 14 MiB                   │
├────────────────┼──────────────────────────┤
│ medium-passes  │ 16                       │
├────────────────┼──────────────────────────┤
│ medium-wearout │ 0.10%                    │
├────────────────┼──────────────────────────┤
│ volume-mounts  │ 7                        │
└────────────────┴──────────────────────────┘
root@backup350:~#

Please look at this GUI reports something diffrent

 

[dcsapak]

seems the drive ids are reversed in your config, and the changer uses id 0 for Naped2 and id 1 for Naped1, which would explain the weird state, because
the part that looks at the changers sees the drive in the "correct" drive, but it's actually in the other one

 

[flowbergit]

How Can I fix it ?

 

[dcsapak]

go to the drive configs and reverse the drive numbers so Naped1 -> 1, and Naped2 -> 0

 

[flowbergit]

2024-01-08T15:26:32+01:00: Starting tape backup job 'BACKUP350:KSSIP:Naped1:KSSIP'
2024-01-08T15:26:32+01:00: update media online status
2024-01-08T15:26:33+01:00: media set uuid: 6bb56fee-1974-4448-9777-cc0e3701c72f
2024-01-08T15:26:33+01:00: found 17 groups
2024-01-08T15:26:33+01:00: backup snapshot "ns/KSSIP/vm/129/2024-01-04T13:59:50Z"
2024-01-08T15:26:38+01:00: allocated new writable media 'BOY020L6'
2024-01-08T15:26:38+01:00: trying to load media 'BOY020L6' into drive 'Naped1'
2024-01-08T15:27:42+01:00: wrong media label BOY023L6 (ac1f7303-4efb-43dc-9082-e047452b557e)
2024-01-08T15:27:42+01:00: Please insert media 'BOY020L6' into changer 'TASMY-KSSIP'

 

[dcsapak]

Ok, now I am confused. Can you provide a bit more information?

Which tape is actually inserted in the drive? (can you get that info e.g. via the changer itself?)
Is the physical label consistent with what is on the tape (you can find that with 'proxmox-tape read-label --drive <drive_name>')

can you also provide info about the changer and drives itself? (e.g. names, models, etc.)

 

[flowbergit]

Currently in Drive is properly Tape with correct BAR CODE but PBS report something strange, in attachment You can see, information about Storage Library

 

[flowbergit]

PBS shows diffrent in difrrent places...

 

[dcsapak]

yeah so maybe because of the confusion before, it wrote the wrong label to the wrong tape
so it thought the label in the drive was BOY023L6 and wrote that there, which now with the correct drive settings it now detects is wrong of course

does it work if you begin with new tapes in a new pool (to test) ?
if that works, i guess the only way is to remove the tapes with the wrong labels from the inventory with 'proxmox-tape media destroy' format and re-label them

 

[flowbergit]

I remove all, recreate a new partion on HP MSL with only 1Drive, I reformat all tapes in pool, and when I try to do backup it's stuck on:

2024-01-17T13:32:19+01:00: Starting tape backup job 'BACKUP350:KSSIP:DRIVE5:KSSIP'
2024-01-17T13:32:19+01:00: update media online status
2024-01-17T13:32:19+01:00: media set uuid: d4655ac0-b7e7-4359-8631-4beae9f0f372
2024-01-17T13:32:20+01:00: found 18 groups
2024-01-17T13:32:20+01:00: backup snapshot "ns/KSSIP/vm/129/2024-01-04T13:59:50Z"

 

[guerby]

I remove all, recreate a new partion on HP MSL with only 1Drive, I reformat all tapes in pool, and when I try to do backup it's stuck on:

2024-01-17T13:32:19+01:00: Starting tape backup job 'BACKUP350:KSSIP:DRIVE5:KSSIP'
2024-01-17T13:32:19+01:00: update media online status
2024-01-17T13:32:19+01:00: media set uuid: d4655ac0-b7e7-4359-8631-4beae9f0f372
2024-01-17T13:32:20+01:00: found 18 groups
2024-01-17T13:32:20+01:00: backup snapshot "ns/KSSIP/vm/129/2024-01-04T13:59:50Z"

Hi,

As I noted here:

https://forum.proxmox.com/threads/t...ort_address-0-not-working.127583/#post-623386

We were just careful to number the drives in proxmox in the same order as displayed by the web UI of the tape library itself which is not the same order as displayed by proxmox-tape drive scan (which seems to sort alphabetically).

"drive1" is proxmox drivenum 0, "drive2" 1, "drive3" 2.

Were you careful to number your drives in proxmox the same way as reported in your tape library own UI ?

 

[chybz]

Hi, same test. same results. "wait_until_ready failed - got timeout"


- Server id HP DL380G8 (from StoreOnce 6500)
- Library is HP ESL G3 with LTO6 Fiber (through SAN switch)
- Library partition has all resources (all slots/i-e/drives)
- 2 drives D09 and D12 are zoned with server

- changer is working from CLI and GUI
- unable to use drive : format / label
- have 2nd configuration on other site : same errors

What did I missed ?

root@bckpcy01srv14:~# proxmox-tape  label --label-text NR0013L6 --drive D09 --pool TESTPOOL
TASK ERROR: open drive 'D09' (/dev/tape/by-id/scsi-80128D0031-sg) failed - wait_until_ready failed - got timeout
Error: task failed (status open drive 'D09' (/dev/tape/by-id/scsi-80128D0031-sg) failed - wait_until_ready failed - got timeout)

root@bckpcy01srv14:/var/log/proxmox-backup/tasks# proxmox-backup-manager versions --verbose
proxmox-backup                unknown  running kernel: 6.8.8-3-pve
proxmox-backup-server         3.2.7-1  running version: 3.2.7     
proxmox-kernel-helper         8.1.0                               
ifupdown2                     unknown                             
libjs-extjs                   7.0.0-4                             
proxmox-backup-docs           3.2.7-1                             
proxmox-backup-client         3.2.7-1                             
proxmox-mail-forward          unknown                             
proxmox-mini-journalreader    1.4.0                               
proxmox-offline-mirror-helper unknown                             
proxmox-widget-toolkit        4.2.3                               
pve-xtermjs                   5.3.0-3                             
smartmontools                 7.3-pve1                           
zfsutils-linux                2.1.11-1