decode element status failed - failed to fill whole buffer

F

flames

Renowned Member
Feb 8, 2018
136
26
68
CH
Hello,

got a used HP MSL2024 changer with an LTO-4 drive and try to setup it up with PBS.
PBS is installed on a HP DL380g8 (2x Xeon 2660v3, 192GB DDR3 ECC, ZFS/Datastore on SSDs).

Following error happens on every occasion. It happens while try to load or unload a tape, starting a backup and also if a backup successfully started, it happens just in the middle of the backup (which instantly cancels).
First tried with PBS 1.0-x, then updated to 2.0-4, error persists.

2021-07-21T00:45:29+02:00: TASK ERROR: error reading element status: decode element status failed - failed to fill whole buffer

Best to reproduce is just try to load/unload a tape, error multiple tries then it will successfully load on a next try. Unload tape, error multiple tries, then it unloads

To get sure it is not a hardware failure, i tested same changer SCSI cable and SCSI controller in a Windows server with Backup Exec.
Still, replaced the SCSI controller by a HP branded Adaptec (PCIe) to get sure its not a firmware issue with the non HP controller. Error persist.

How could i collect more info? Is there a debug/verbose mode?

Thanks in advance.
 
  • Like
Reactions: Tobias_Seitz
this is maybe a bug. we recently encountered a similar behaviour on aws tape gateway, and are on it to fix it... i'll write here if i sent a patch to the devel list
 
  • Like
Reactions: Tobias_Seitz and flames
  • Like
Reactions: flames
Thank you Dominik!
there is no such file on installation, do i need to compile from source? or will the patch be packaged in the testing repository?
 
Last edited:
flames said:
there is no such file on installation, do i need to compile from source? or will the patch be packaged in the testing repository?
Click to expand...
yes you'd have to compile from source ( a better fix than i posted above is already commited to git), but the version was not bumped yet
when that happens, normally soon after the package will get uploaded to the testing repository
 
  • Like
Reactions: flames
i am failing to build the package from git master branch (including all your todays commits). do i need an older lexical-core library to build successfully? not a dev here and especially rust is new for me :)

Debian 11, Kernel 5.11.22-4
PBS devel sorurce, updated + installed all dependencies

# cargo build
Compiling libc v0.2.94
Compiling proc-macro2 v1.0.26
Compiling unicode-xid v0.2.0
Compiling syn v1.0.70
Compiling autocfg v1.0.1
Compiling cfg-if v0.1.10
Compiling cfg-if v1.0.0
Compiling memchr v2.3.3
Compiling log v0.4.11
Compiling pkg-config v0.3.18
Compiling version_check v0.9.2
Compiling lazy_static v1.4.0
Compiling cc v1.0.68
Compiling bitflags v1.2.1
Compiling ryu v1.0.2
Compiling smallvec v1.6.1
Compiling serde v1.0.106
Compiling pin-project-lite v0.2.6
Compiling bytes v1.0.0
Compiling once_cell v1.5.2
Compiling scopeguard v1.1.0
Compiling proc-macro-hack v0.5.19
Compiling unicode-width v0.1.8
Compiling itoa v0.4.3
Compiling semver-parser v0.7.0
Compiling futures-core v0.3.14
Compiling proc-macro-nested v0.1.3
Compiling regex-syntax v0.6.17
Compiling futures-sink v0.3.14
Compiling pin-utils v0.1.0
Compiling futures-io v0.3.14
Compiling futures-task v0.3.14
Compiling slab v0.4.2
Compiling fnv v1.0.6
Compiling matches v0.1.8
Compiling arrayvec v0.5.2
Compiling static_assertions v1.1.0
Compiling openssl v0.10.32
Compiling anyhow v1.0.31
Compiling foreign-types-shared v0.1.1
Compiling percent-encoding v2.1.0
Compiling base64 v0.12.1
Compiling glob v0.3.0
Compiling hashbrown v0.11.2
Compiling httparse v1.4.0
Compiling try-lock v0.2.2
Compiling httpdate v1.0.0
Compiling tower-service v0.3.0
Compiling unicode-segmentation v1.6.0
Compiling either v1.5.0
Compiling utf8parse v0.2.0
Compiling humantime v2.0.0
Compiling base32 v0.4.0
Compiling termcolor v1.1.0
Compiling vec_map v0.8.1
Compiling strsim v0.9.3
Compiling ansi_term v0.12.1
Compiling bindgen v0.55.1
Compiling peeking_take_while v0.1.2
Compiling lazycell v1.3.0
Compiling getrandom v0.1.13
Compiling rustc-hash v1.1.0
Compiling shlex v0.1.1
Compiling crc32fast v1.2.0
Compiling typenum v1.12.0
Compiling pbs-buildcfg v2.0.6 (/root/pbssrc/proxmox-backup/pbs-buildcfg)
Compiling adler32 v1.2.0
Compiling same-file v1.0.6
Compiling ppv-lite86 v0.2.6
Compiling backtrace-sys v0.1.35
Compiling ucd-trie v0.1.3
Compiling curl v0.4.36
Compiling zstd-safe v3.0.0+zstd.1.4.8
Compiling block-padding v0.2.1
Compiling cpuid-bool v0.1.2
Compiling maplit v1.0.1
Compiling openssl-probe v0.1.2
Compiling opaque-debug v0.3.0
Compiling untrusted v0.7.1
Compiling rustc-demangle v0.1.16
Compiling spin v0.5.2
Compiling siphasher v0.3.1
Compiling pam-sys v0.5.6
Compiling webauthn-rs v0.2.5
Compiling half v1.6.0
Compiling xdg v2.2.0
Compiling quick-error v1.2.3
Compiling instant v0.1.9
Compiling thread_local v1.0.1
Compiling tracing-core v0.1.17
Compiling lock_api v0.4.2
Compiling textwrap v0.11.0
Compiling futures-channel v0.3.14
Compiling unicode-bidi v0.3.4
Compiling foreign-types v0.3.2
Compiling semver v0.9.0
Compiling unicode-normalization v0.1.12
Compiling form_urlencoded v1.0.1
Compiling itertools v0.9.0
Compiling http v0.2.2
Compiling miniz_oxide v0.3.6
Compiling walkdir v2.3.1
Compiling nom v5.1.1
Compiling generic-array v0.14.4
Compiling nom v4.2.3
Compiling error-chain v0.12.4
Compiling tokio v1.6.1
Compiling indexmap v1.6.2
Compiling num-traits v0.2.14
Compiling num-integer v0.1.42
Compiling crossbeam-utils v0.8.1
Compiling pest v2.1.2
Compiling libudev-sys v0.1.4
Compiling clang-sys v1.0.1
Compiling rustc_version v0.2.3
Compiling tracing v0.1.22
Compiling want v0.3.0
Compiling aho-corasick v0.7.10
Compiling lexical-core v0.6.7
Compiling quote v1.0.7
Compiling idna v0.2.0
Compiling openssl-sys v0.9.60
Compiling curl-sys v0.4.42+curl-7.76.0
Compiling ring v0.16.20
Compiling apt-pkg-native v0.3.2
Compiling pest_meta v2.1.1
Compiling crossbeam-channel v0.5.0
Compiling regex v1.3.7
Compiling digest v0.9.0
Compiling block-buffer v0.9.0
Compiling ordered-float v2.5.1
Compiling sha2 v0.9.2
error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/bhcomp.rs:62:24
|
62 | let bytes = bits / Limb::BITS;
| ^^^^^^^^^^ expected `usize`, found `u32`

error[E0277]: cannot divide `usize` by `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/bhcomp.rs:62:22
|
62 | let bytes = bits / Limb::BITS;
| ^ no implementation for `usize / u32`
|
= help: the trait `Div<u32>` is not implemented for `usize`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/bigcomp.rs:157:55
|
157 | let nlz = den.leading_zeros().wrapping_sub(wlz) & (u32::BITS - 1);
| ^^^^^^^^^^^^^^^ expected `usize`, found `u32`

error[E0277]: no implementation for `usize & u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/bigcomp.rs:157:53
|
157 | let nlz = den.leading_zeros().wrapping_sub(wlz) & (u32::BITS - 1);
| ^ no implementation for `usize & u32`
|
= help: the trait `BitAnd<u32>` is not implemented for `usize`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/bigcomp.rs:175:40
|
175 | let (q, r) = shift.ceil_divmod(Limb::BITS);
| ^^^^^^^^^^ expected `usize`, found `u32`
|
help: you can convert a `u32` to a `usize` and panic if the converted value doesn't fit
|
175 | let (q, r) = shift.ceil_divmod(Limb::BITS.try_into().unwrap());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Compiling parking_lot_core v0.8.0
Compiling signal-hook-registry v1.3.0
Compiling mio v0.7.6
Compiling num_cpus v1.13.0
Compiling socket2 v0.4.0
Compiling dirs-sys-next v0.1.1
Compiling atty v0.2.14
Compiling fs2 v0.4.3
Compiling nix v0.19.1
Compiling which v3.0.0
Compiling flate2 v1.0.13
Compiling time v0.1.42
error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1049:42
|
1049 | let mut count = index.saturating_mul(Limb::BITS);
| ^^^^^^^^^^ expected `usize`, found `u32`
|
help: you can convert a `u32` to a `usize` and panic if the converted value doesn't fit
|
1049 | let mut count = index.saturating_mul(Limb::BITS.try_into().unwrap());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1064:28
|
1064 | Limb::BITS.checked_mul(x.len())
| ^^^^^^^ expected `u32`, found `usize`
|
help: you can convert a `usize` to a `u32` and panic if the converted value doesn't fit
|
1064 | Limb::BITS.checked_mul(x.len().try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1065:22
|
1065 | .map(|v| v - nlz)
| ^^^ expected `u32`, found `usize`

error[E0277]: cannot subtract `usize` from `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1065:20
|
1065 | .map(|v| v - nlz)
| ^ no implementation for `u32 - usize`
|
= help: the trait `Sub<usize>` is not implemented for `u32`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1066:20
|
1066 | .unwrap_or(usize::max_value())
| ^^^^^^^^^^^^^^^^^^ expected `u32`, found `usize`
|
help: you can convert a `usize` to a `u32` and panic if the converted value doesn't fit
|
1066 | .unwrap_or(usize::max_value().try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1064:5
|
1060 | pub fn bit_length(x: &[Limb]) -> usize {
| ----- expected `usize` because of return type
...
1064 | / Limb::BITS.checked_mul(x.len())
1065 | | .map(|v| v - nlz)
1066 | | .unwrap_or(usize::max_value())
| |______________________________________^ expected `usize`, found `u32`
|
help: you can convert a `u32` to a `usize` and panic if the converted value doesn't fit
|
1064 | Limb::BITS.checked_mul(x.len())
1065 | .map(|v| v - nlz)
1066 | .unwrap_or(usize::max_value()).try_into().unwrap()
|

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1091:23
|
1091 | debug_assert!(n < bits && n != 0);
| ^^^^ expected `usize`, found `u32`
|
help: you can convert a `u32` to a `usize` and panic if the converted value doesn't fit
|
1091 | debug_assert!(n < bits.try_into().unwrap() && n != 0);
| ^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1098:25
|
1098 | let lshift = bits - n;
| ^ expected `u32`, found `usize`

error[E0277]: cannot subtract `usize` from `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1098:23
|
1098 | let lshift = bits - n;
| ^ no implementation for `u32 - usize`
|
= help: the trait `Sub<usize>` is not implemented for `u32`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1140:19
|
1140 | let rem = n % bits;
| ^^^^ expected `usize`, found `u32`

error[E0277]: cannot mod `usize` by `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1140:17
|
1140 | let rem = n % bits;
| ^ no implementation for `usize % u32`
|
= help: the trait `Rem<u32>` is not implemented for `usize`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1141:19
|
1141 | let div = n / bits;
| ^^^^ expected `usize`, found `u32`

error[E0277]: cannot divide `usize` by `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1141:17
|
1141 | let div = n / bits;
| ^ no implementation for `usize / u32`
|
= help: the trait `Div<u32>` is not implemented for `usize`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1197:23
|
1197 | debug_assert!(n < bits);
| ^^^^ expected `usize`, found `u32`
|
help: you can convert a `u32` to a `usize` and panic if the converted value doesn't fit
|
1197 | debug_assert!(n < bits.try_into().unwrap());
| ^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1207:25
|
1207 | let rshift = bits - n;
| ^ expected `u32`, found `usize`

error[E0277]: cannot subtract `usize` from `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1207:23
|
1207 | let rshift = bits - n;
| ^ no implementation for `u32 - usize`
|
= help: the trait `Sub<usize>` is not implemented for `u32`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1259:19
|
1259 | let rem = n % bits;
| ^^^^ expected `usize`, found `u32`

error[E0277]: cannot mod `usize` by `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1259:17
|
1259 | let rem = n % bits;
| ^ no implementation for `usize % u32`
|
= help: the trait `Rem<u32>` is not implemented for `usize`

error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1260:19
|
1260 | let div = n / bits;
| ^^^^ expected `usize`, found `u32`

error[E0277]: cannot divide `usize` by `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:1260:17
|
1260 | let div = n / bits;
| ^ no implementation for `usize / u32`
|
= help: the trait `Div<u32>` is not implemented for `usize`

Compiling pathpatterns v0.1.2
error[E0308]: mismatched types
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:2071:27
|
2071 | let rs = Limb::BITS - s;
| ^ expected `u32`, found `usize`

error[E0277]: cannot subtract `usize` from `u32`
--> /usr/share/cargo/registry/lexical-core-0.6.7/src/atof/algorithm/math.rs:2071:25
|
2071 | let rs = Limb::BITS - s;
| ^ no implementation for `u32 - usize`
|
= help: the trait `Sub<usize>` is not implemented for `u32`

error: aborting due to 27 previous errors

Some errors have detailed explanations: E0277, E0308.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `lexical-core`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
 
Last edited:
never mind, downgraded rustc to 1.52.1, now compilation is was successfull. building a package now.
Code: 
rustup install 1.52.1
export RUSTUP_TOOLCHAIN=1.52.1
source "$HOME/.cargo/env"
 
Last edited:
Package built and installed. Now i have for the first time a successfull job (cleaning drive).
But, there is now this error instead: 2021-07-22T20:04:58+02:00: TASK ERROR: worker panicked: unknown type.
This error happens now with ~same frequency, as the one before, while before i could not run any successful task at all, now cleaning job is after some retries successfully finished.

/var/log/proxmox-backup/tasks

Code: 
UPID:pbs01:00246257:00AA55CC:00000000:60F9B34A:clean-drive:lto4:root@pam: 60F9B34A worker panicked: unknown type.
UPID:pbs01:00246257:00AA55CC:00000001:60F9B39C:clean-drive:lto4:root@pam: 60F9B39D worker panicked: unknown type.
UPID:pbs01:00246257:00AA55CC:00000002:60F9B3A2:clean-drive:lto4:root@pam: 60F9B429 OK

and now there is also more in /var/log/syslog (previously there was nothing relevant to the task/error)...

Code: 
Jul 22 20:04:23 pbs01 systemd[1]: Reloading.
Jul 22 20:04:23 pbs01 systemd[1]: proxmox-backup-daily-update.timer: Succeeded.
Jul 22 20:04:23 pbs01 systemd[1]: Stopped Daily Proxmox Backup Server update and maintenance activities.
Jul 22 20:04:23 pbs01 systemd[1]: Stopping Daily Proxmox Backup Server update and maintenance activities.
Jul 22 20:04:23 pbs01 systemd[1]: Started Daily Proxmox Backup Server update and maintenance activities.
Jul 22 20:04:24 pbs01 systemd[1]: Reloading.
Jul 22 20:04:25 pbs01 systemd[1]: Reloading Proxmox Backup API Server.
Jul 22 20:04:25 pbs01 proxmox-backup-api[1908705]: got reload request (SIGHUP)
Jul 22 20:04:25 pbs01 proxmox-backup-api[1908705]: SET SHUTDOWN MODE
Jul 22 20:04:25 pbs01 proxmox-backup-api[1908705]: SET SHUTDOWN MODE
Jul 22 20:04:25 pbs01 proxmox-backup-api[1908705]: daemon reload...
Jul 22 20:04:25 pbs01 systemd[1]: Reloaded Proxmox Backup API Server.
Jul 22 20:04:25 pbs01 systemd[1]: Reloading Proxmox Backup API Proxy Server.
Jul 22 20:04:25 pbs01 proxmox-backup-proxy[1908771]: got reload request (SIGHUP)
Jul 22 20:04:25 pbs01 proxmox-backup-proxy[1908771]: SET SHUTDOWN MODE
Jul 22 20:04:25 pbs01 proxmox-backup-proxy[1908771]: daemon reload...
Jul 22 20:04:25 pbs01 systemd[1]: Reloaded Proxmox Backup API Proxy Server.
Jul 22 20:04:26 pbs01 systemd[1]: proxmox-backup.service: Supervising process 2384469 which is not our child. We'll most likely not notice when it exits.
Jul 22 20:04:26 pbs01 systemd[1]: proxmox-backup-proxy.service: Supervising process 2384471 which is not our child. We'll most likely not notice when it exits.
Jul 22 20:04:27 pbs01 proxmox-backup-api[1908705]: daemon shut down...
Jul 22 20:04:27 pbs01 proxmox-backup-api[1908705]: server shutting down, waiting for active workers to complete
Jul 22 20:04:27 pbs01 proxmox-backup-api[1908705]: done - exit server
Jul 22 20:04:27 pbs01 proxmox-backup-proxy[1908771]: daemon shut down...
Jul 22 20:04:27 pbs01 proxmox-backup-proxy[1908771]: server shutting down, waiting for active workers to complete
Jul 22 20:04:27 pbs01 proxmox-backup-proxy[1908771]: done - exit server
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jul 22 20:04:40 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:40 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:04:40 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:04:47 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:47 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:04:47 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:04:58 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000000:60F9B34A:clean-drive:lto4:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:58 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:04:58 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:05:50 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:05:50 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:05:50 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:06:12 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:06:12 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:06:12 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:06:21 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000001:60F9B39C:clean-drive:lto4:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:06:21 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:06:21 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:07:05 pbs01 systemd[1]: Started Session 43 of user root.
Jul 22 20:07:05 pbs01 systemd[1]: session-40.scope: Succeeded.
Jul 22 20:07:10 pbs01 systemd[1]: Started Session 44 of user root.
Jul 22 20:08:41 pbs01 kernel: [111890.404242] st 1:0:4:0: Mode parameters changed
Jul 22 20:08:51 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:08:51 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:08:51 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
 
the panic happens less often, than the buffer error before. in the changer menu clicking reload it was previously 3-4 times error, one successful. now 1-2 succesfull, one panic.
i will run a backup job now, and see, if will finish successfully. could take a while (12TB, LTO-4...)
are those issues really a bug, or is it because of LTO-4 (support "best effort")?

can't start a the backup job....
Jul 22 20:04:27 pbs01 proxmox-backup-proxy[1908771]: done - exit server
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:04:37 pbs01 proxmox-backup-proxy[2384471]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jul 22 20:04:40 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:40 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:04:40 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:04:47 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:47 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:04:47 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:04:58 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000000:60F9B34A:clean-drive:lto4:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:04:58 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:04:58 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:05:50 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:05:50 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:05:50 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:06:12 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:06:12 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:06:12 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:06:21 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000001:60F9B39C:clean-drive:lto4:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:06:21 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:06:21 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:07:05 pbs01 systemd[1]: Started Session 43 of user root.
Jul 22 20:07:05 pbs01 systemd[1]: session-40.scope: Succeeded.
Jul 22 20:07:10 pbs01 systemd[1]: Started Session 44 of user root.
Jul 22 20:08:41 pbs01 kernel: [111890.404242] st 1:0:4:0: Mode parameters changed
Jul 22 20:08:51 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:08:51 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:08:51 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:16:35 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:16:35 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:16:35 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:16:37 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:16:37 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:16:37 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:17:01 pbs01 CRON[2393577]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jul 22 20:19:32 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:19:32 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:19:32 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:19:35 pbs01 proxmox-backup-proxy[2384471]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 22 20:19:35 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:19:35 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:19:49 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000003:60F9B6C5:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:19:49 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:19:49 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: no empty media in pool, try to reuse expired media
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS005L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS004L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS002L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS006L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS009L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS003L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS008L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS007L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS011L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS010L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS001L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: reuse expired media 'PBS009L4'
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000004:60F9B6CC:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:19:56 pbs01 proxmox-backup-proxy[2384471]: could not send chunk to reader thread: sending on a closed channel
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: no empty media in pool, try to reuse expired media
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS005L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS004L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS002L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS006L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS009L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS003L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS008L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS007L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS011L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS010L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS001L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: reuse expired media 'PBS008L4'
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000005:60F9B6D1:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:20:02 pbs01 proxmox-backup-proxy[2384471]: could not send chunk to reader thread: sending on a closed channel
Jul 22 20:20:07 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000006:60F9B6D7:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:20:07 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:20:07 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: no empty media in pool, try to reuse expired media
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS005L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS004L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS002L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS006L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS009L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS003L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS008L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS007L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS011L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS010L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS001L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: reuse expired media 'PBS001L4'
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000007:60F9B6DB:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:20:11 pbs01 proxmox-backup-proxy[2384471]: could not send chunk to reader thread: sending on a closed channel
Jul 22 20:20:16 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000008:60F9B6E0:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:20:16 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:20:16 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: no empty media in pool, try to reuse expired media
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS005L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS004L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS002L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS006L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS009L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS003L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS008L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS007L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS011L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS010L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS001L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: reuse expired media 'PBS002L4'
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:00000009:60F9B6E4:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: left: `0`,
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:20:21 pbs01 proxmox-backup-proxy[2384471]: could not send chunk to reader thread: sending on a closed channel
 
Last edited:
task log
Code: 
2021-07-22T20:24:56+02:00: Starting tape backup job 'ait:dailyfull:lto4:daily'
2021-07-22T20:24:56+02:00: update media online status
2021-07-22T20:24:56+02:00: starting new media set - reason: policy is AlwaysCreate
2021-07-22T20:24:56+02:00: media set uuid: 42eec466-daee-4874-b003-a1b40caea85f
2021-07-22T20:24:56+02:00: found 54 groups
2021-07-22T20:24:56+02:00: backup snapshot ct/151/2021-07-01T20:00:02Z
2021-07-22T20:24:56+02:00: allocated new writable media 'PBS003L4'
2021-07-22T20:24:56+02:00: loading media 'PBS003L4' into drive 'lto4'
2021-07-22T20:24:56+02:00: TASK ERROR: worker panicked: unknown type.

syslog
Code: 
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: no empty media in pool, try to reuse expired media
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS005L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS004L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS002L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS006L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS009L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS003L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS008L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS007L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS011L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS010L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: found expired media on media 'PBS001L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: reuse expired media 'PBS003L4'
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: thread 'UPID:pbs01:00246257:00AA55CC:0000000A:60F9B7F8:tape-backup-job:ait\x3adailyfull\x3alto4\x3adaily:root@pam:' panicked at 'assertion failed: `(left != right)`
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]:   left: `0`,
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]:  right: `0`', /rustc/88f19c6dab716c6281af7602e30f413e809c5974/library/core/src/slice/mod.rs:863:9
Jul 22 20:24:56 pbs01 proxmox-backup-proxy[2384471]: could not send chunk to reader thread: sending on a closed channel
 
i do not know what exactly is the problem, possibly that you did not compile it with out toolchain but rustup?
threads panicking is not something that should happen on proper built binaries
(maybe some incompatibilty with crates? idk )

i'd recommend either waiting on the built packages, or starting from scratch but using our toolchain
see https://pbs.proxmox.com/wiki/index.php/Developer_Documentation for build instructions too

NOTE: instructions are still for 'buster', most could be simply replaced by 'bullseye' to build for pbs 2.0
if you have any problems with building with our toolchain instead of rustup, don't hesitate to ask
 
  • Like
Reactions: flames
thanks! i suspected, that i was doing something wrong or have not fullfilled all needed dependencies. so i removed all rustc and dependencies which i installed previously, then recompiled exactly as in the documentation and build notes you posted above. still got same error.
now i have seen, that the 2.0.7-1 is already in the testing repository and decided to remove the selfcompiled version, installed from repository, and strangely same error.
Code: 
Jul 23 11:19:04 pbs01 proxmox-backup-proxy[1196419]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 23 11:19:04 pbs01 proxmox-backup-proxy[1196419]:   left: `0`,
Jul 23 11:19:04 pbs01 proxmox-backup-proxy[1196419]:  right: `0`', /usr/src/rustc-1.52.1/library/core/src/slice/mod.rs:863:9
Jul 23 11:19:04 pbs01 proxmox-backup-proxy[1196419]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jul 23 11:19:15 pbs01 proxmox-backup-proxy[1196419]: thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
Jul 23 11:19:15 pbs01 proxmox-backup-proxy[1196419]:   left: `0`,
Jul 23 11:19:15 pbs01 proxmox-backup-proxy[1196419]:  right: `0`', /usr/src/rustc-1.52.1/library/core/src/slice/mod.rs:863:9
 
Last edited:
mhmm.. looking into it and what changed, it would mean that your changer does not return any descriptors

can you install the package 'proxmox-backup-server-dbgsym' and then post the output of
Code: 
pmtx status --changer <YOUR_CHANGERNAME>
?

edit: can you also post the output of
Code: 
sg_raw -r 64k <PATH_TO_CHANGER_DEVICE> B8 11 00 00 ff ff 00 ff ff ff 00 00
sg_raw -r 64k <PATH_TO_CHANGER_DEVICE> B8 12 00 00 ff ff 00 ff ff ff 00 00
sg_raw -r 64k <PATH_TO_CHANGER_DEVICE> B8 13 00 00 ff ff 00 ff ff ff 00 00
sg_raw -r 64k <PATH_TO_CHANGER_DEVICE> B8 14 00 00 ff ff 00 ff ff ff 00 00
 
Last edited:
  • Like
Reactions: flames
Code: 
# pmtx status --changer msl2024
using device /dev/tape/by-id/scsi-DEC80205XR
Transport Element (Griper)      0: Empty
Data Transfer Element (Drive)   0: Empty, Serial: HUE08455BM
  Storage Element   1: VolumeTag("PBS001L4")
  Storage Element   2: VolumeTag("PBS002L4")
  Storage Element   3: VolumeTag("PBS003L4")
  Storage Element   4: VolumeTag("PBS004L4")
  Storage Element   5: VolumeTag("PBS005L4")
  Storage Element   6: VolumeTag("PBS006L4")
  Storage Element   7: VolumeTag("PBS007L4")
  Storage Element   8: VolumeTag("PBS008L4")
  Storage Element   9: VolumeTag("PBS009L4")
  Storage Element  10: VolumeTag("PBS010L4")
  Storage Element  11: VolumeTag("PBS011L4")
  Storage Element  12: Empty
  Storage Element  13: Empty
  Storage Element  14: Empty
  Storage Element  15: Empty
  Storage Element  16: Empty
  Storage Element  17: Empty
  Storage Element  18: Empty
  Storage Element  19: Empty
  Storage Element  20: Empty
  Storage Element  21: Empty
  Storage Element  22: Empty
  Storage Element  23: Empty
  Storage Element  24: VolumeTag("CLN001CU")

and after issueing the above command multiple times, it was 3 times successfull, on fourth time it returned:
Code: 
# pmtx status --changer msl2024
using device /dev/tape/by-id/scsi-DEC80205XR
thread 'main' panicked at 'assertion failed: `(left != right)`
  left: `0`,
 right: `0`', /usr/src/rustc-1.52.1/library/core/src/slice/mod.rs:863:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 
Last edited:
ah could you issue the command from my edit of the previous post?
and also try to execute the 'pmtx' command with 'RUST_BACKTRACE=1' in front of it like so:

Code: 
RUST_BACKTRACE=1 pmtx status --changer msl2024

until it errors out ?
 
  • Like
Reactions: flames
root@pbs01:~/pbssrc/proxmox-backup# RUST_BACKTRACE=1 pmtx status --changer msl2024
using device /dev/tape/by-id/scsi-DEC80205XR
Transport Element (Griper) 0: Empty
Data Transfer Element (Drive) 0: Empty, Serial: HUE08455BM
Storage Element 1: VolumeTag("PBS001L4")
Storage Element 2: VolumeTag("PBS002L4")
Storage Element 3: VolumeTag("PBS003L4")
Storage Element 4: VolumeTag("PBS004L4")
Storage Element 5: VolumeTag("PBS005L4")
Storage Element 6: VolumeTag("PBS006L4")
Storage Element 7: VolumeTag("PBS007L4")
Storage Element 8: VolumeTag("PBS008L4")
Storage Element 9: VolumeTag("PBS009L4")
Storage Element 10: VolumeTag("PBS010L4")
Storage Element 11: VolumeTag("PBS011L4")
Storage Element 12: Empty
Storage Element 13: Empty
Storage Element 14: Empty
Storage Element 15: Empty
Storage Element 16: Empty
Storage Element 17: Empty
Storage Element 18: Empty
Storage Element 19: Empty
Storage Element 20: Empty
Storage Element 21: Empty
Storage Element 22: Empty
Storage Element 23: Empty
Storage Element 24: VolumeTag("CLN001CU")
root@pbs01:~/pbssrc/proxmox-backup# RUST_BACKTRACE=1 pmtx status --changer msl2024
using device /dev/tape/by-id/scsi-DEC80205XR
thread 'main' panicked at 'assertion failed: `(left != right)`
left: `0`,
right: `0`', /usr/src/rustc-1.52.1/library/core/src/slice/mod.rs:863:9
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
root@pbs01:~/pbssrc/proxmox-backup# sg_raw -r 64k /dev/tape/by-id/scsi-DEC80205XR B8 11 00 00 ff ff 00 ff ff ff 00 00
SCSI Status: Good

Received 68 bytes of data:
00 00 00 00 01 00 00 00 3c 01 80 00 34 00 00 00 34 .......<...4...4
10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
40 00 00 00 00 ....
root@pbs01:~/pbssrc/proxmox-backup# sg_raw -r 64k /dev/tape/by-id/scsi-DEC80205XR B8 12 00 00 ff ff 00 ff ff ff 00 00
SCSI Status: Good

Received 1264 bytes of data:
00 03 e9 00 18 00 00 04 e8 02 80 00 34 00 00 04 e0 ...........4....
10 03 e9 09 00 00 00 00 00 00 81 03 e9 50 42 53 30 ............PBS0
20 30 31 4c 34 20 20 20 20 20 20 20 20 20 20 20 20 01L4
30 20 20 20 20 20 20 20 20 20 20 20 00 00 00 00 00 .....
40 00 00 00 00 03 ea 09 00 00 00 00 00 00 81 03 ea ................
50 50 42 53 30 30 32 4c 34 20 20 20 20 20 20 20 20 PBS002L4
60 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 00 .
70 00 00 00 00 00 00 00 00 03 eb 09 00 00 00 00 00 ................
80 00 81 03 eb 50 42 53 30 30 33 4c 34 20 20 20 20 ....PBS003L4
90 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
a0 20 20 20 00 00 00 00 00 00 00 00 00 03 ec 09 00 .............
b0 00 00 00 00 00 81 03 ec 50 42 53 30 30 34 4c 34 ........PBS004L4
c0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
d0 20 20 20 20 20 20 20 00 00 00 00 00 00 00 00 00 .........
e0 03 ed 09 00 00 00 00 00 00 81 03 ed 50 42 53 30 ............PBS0
f0 30 35 4c 34 20 20 20 20 20 20 20 20 20 20 20 20 05L4
100 20 20 20 20 20 20 20 20 20 20 20 00 00 00 00 00 .....
110 00 00 00 00 03 ee 09 00 00 00 00 00 00 81 03 ee ................
120 50 42 53 30 30 36 4c 34 20 20 20 20 20 20 20 20 PBS006L4
130 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 00 .
140 00 00 00 00 00 00 00 00 03 ef 09 00 00 00 00 00 ................
150 00 81 03 ef 50 42 53 30 30 37 4c 34 20 20 20 20 ....PBS007L4
160 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
170 20 20 20 00 00 00 00 00 00 00 00 00 03 f0 09 00 .............
180 00 00 00 00 00 81 03 f0 50 42 53 30 30 38 4c 34 ........PBS008L4
190 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
1a0 20 20 20 20 20 20 20 00 00 00 00 00 00 00 00 00 .........
1b0 03 f1 09 00 00 00 00 00 00 81 03 f1 50 42 53 30 ............PBS0
1c0 30 39 4c 34 20 20 20 20 20 20 20 20 20 20 20 20 09L4
1d0 20 20 20 20 20 20 20 20 20 20 20 00 00 00 00 00 .....
1e0 00 00 00 00 03 f2 09 00 00 00 00 00 00 81 03 f2 ................
1f0 50 42 53 30 31 30 4c 34 20 20 20 20 20 20 20 20 PBS010L4
200 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 00 .
210 00 00 00 00 00 00 00 00 03 f3 09 00 00 00 00 00 ................
220 00 81 03 f3 50 42 53 30 31 31 4c 34 20 20 20 20 ....PBS011L4
230 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
240 20 20 20 00 00 00 00 00 00 00 00 00 03 f4 08 00 .............
250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
270 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
280 03 f5 08 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
290 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2b0 00 00 00 00 03 f6 08 00 00 00 00 00 00 00 00 00 ................
2c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
2e0 00 00 00 00 00 00 00 00 03 f7 08 00 00 00 00 00 ................
2f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
300 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
310 00 00 00 00 00 00 00 00 00 00 00 00 03 f8 08 00 ................
320 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
330 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
340 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
350 03 f9 08 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
360 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
370 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
380 00 00 00 00 03 fa 08 00 00 00 00 00 00 00 00 00 ................
390 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3b0 00 00 00 00 00 00 00 00 03 fb 08 00 00 00 00 00 ................
3c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
3e0 00 00 00 00 00 00 00 00 00 00 00 00 03 fc 08 00 ................
3f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
400 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
420 03 fd 08 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
430 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
440 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
450 00 00 00 00 03 fe 08 00 00 00 00 00 00 00 00 00 ................
460 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
470 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
480 00 00 00 00 00 00 00 00 03 ff 08 00 00 00 00 00 ................
490 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
4a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
4b0 00 00 00 00 00 00 00 00 00 00 00 00 04 00 09 00 ................
4c0 00 00 00 00 00 82 04 00 43 4c 4e 30 30 31 43 55 ........CLN001CU
4d0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
4e0 20 20 20 20 20 20 20 00 00 00 00 00 00 00 00 00 .........
root@pbs01:~/pbssrc/proxmox-backup# sg_raw -r 64k /dev/tape/by-id/scsi-DEC80205XR B8 13 00 00 ff ff 00 ff ff ff 00 00
SCSI Status: Good

Received 8 bytes of data:
00 00 65 00 00 00 00 00 00 .e......
root@pbs01:~/pbssrc/proxmox-backup# sg_raw -r 64k /dev/tape/by-id/scsi-DEC80205XR B8 14 00 00 ff ff 00 ff ff ff 00 00
SCSI Status: Good

Received 68 bytes of data:
00 00 01 00 01 00 00 00 3c 04 80 00 34 00 00 00 34 .......<...4...4
10 00 01 08 00 00 00 20 04 00 00 00 00 00 00 00 00 ...... .........
20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
40 00 00 00 00 ....
root@pbs01:~/pbssrc/proxmox-backup#
 
hi thanks for the output.

did you install the dbgsym package like i said earlier? if not, please do that and run it again with `RUST_BACKTRACE=full`
e.g.
Code: 
RUST_BACKTRACE=full pmtx status --changer msl2024

edit: i am trying to find out where exactly the code panics to preven that, but it seems that sometimes the library returns bogus data?
 
Last edited:
  • Like
Reactions: flames
yes, the package is installed. indeed, it looks like the tape library has some issues.
root@pbs01:~/pbssrc/proxmox-backup# RUST_BACKTRACE=full pmtx status --changer msl2024
using device /dev/tape/by-id/scsi-DEC80205XR
Transport Element (Griper) 0: Empty
Data Transfer Element (Drive) 0: Empty, Serial: HUE08455BM
Storage Element 1: VolumeTag("PBS001L4")
Storage Element 2: VolumeTag("PBS002L4")
Storage Element 3: VolumeTag("PBS003L4")
Storage Element 4: VolumeTag("PBS004L4")
Storage Element 5: VolumeTag("PBS005L4")
Storage Element 6: VolumeTag("PBS006L4")
Storage Element 7: VolumeTag("PBS007L4")
Storage Element 8: VolumeTag("PBS008L4")
Storage Element 9: VolumeTag("PBS009L4")
Storage Element 10: VolumeTag("PBS010L4")
Storage Element 11: VolumeTag("PBS011L4")
Storage Element 12: Empty
Storage Element 13: Empty
Storage Element 14: Empty
Storage Element 15: Empty
Storage Element 16: Empty
Storage Element 17: Empty
Storage Element 18: Empty
Storage Element 19: Empty
Storage Element 20: Empty
Storage Element 21: Empty
Storage Element 22: Empty
Storage Element 23: Empty
Storage Element 24: VolumeTag("CLN001CU")
root@pbs01:~/pbssrc/proxmox-backup# RUST_BACKTRACE=full pmtx status --changer msl2024
using device /dev/tape/by-id/scsi-DEC80205XR
thread 'main' panicked at 'assertion failed: `(left != right)`
left: `0`,
right: `0`', /usr/src/rustc-1.52.1/library/core/src/slice/mod.rs:863:9
stack backtrace:
0: 0x55e8411abf4c - <unknown>
1: 0x55e8411ddccf - <unknown>
2: 0x55e8411abc86 - <unknown>
3: 0x55e8411bc4d5 - <unknown>
4: 0x55e8411bc0a3 - <unknown>
5: 0x55e8411bcb15 - <unknown>
6: 0x55e8411ac3d7 - <unknown>
7: 0x55e8411ac08c - <unknown>
8: 0x55e8411bc6b2 - <unknown>
9: 0x55e8411e0131 - <unknown>
10: 0x55e8411e025f - <unknown>
11: 0x55e8410ecc7b - <unknown>
12: 0x55e8410f9fc7 - <unknown>
13: 0x55e8410de139 - <unknown>
14: 0x55e8410df93a - <unknown>
15: 0x55e8410d358b - <unknown>
16: 0x55e8410cf1ef - <unknown>
17: 0x55e841118a2f - <unknown>
18: 0x55e84111a084 - <unknown>
19: 0x55e8410d6ec8 - <unknown>
20: 0x55e8410d16d0 - <unknown>
21: 0x55e8410da453 - <unknown>
22: 0x55e8410da46d - <unknown>
23: 0x55e8411be281 - <unknown>
24: 0x55e8410d5318 - <unknown>
25: 0x7fdce5f32d0a - __libc_start_main
26: 0x55e8410cd0fa - <unknown>
27: 0x0 - <unknown>

if it would help, i can add your ssh keys and open it in the firewall. but the question is, is it worth to waste your time on that? for me this is not productive, just play to learn (and maybe for later use at work). if this is interesting for pbs tape implementation, then we can continue. if the issue is more likely a broken hardware, then trash it would be the better option. i did not assumed any hardware issue, because it works flawlessly with other software (tested with veeam trial in the meantime). you decide :)
 
Last edited:
mhmm... from the data you sent does not really tell if it's a hardware problem or not. while it looks like the library sends bogus data, there still might be something wrong with our code. (maybe
the library just behaves strangely in a way we do not anticipate?)

remote ssh debugging is reserved for subscribers with level standard or premium, so i do not think this is an option here...
but we can try a few things:
first debugging the panic:

you can install 'gdb' and run the pmtx command with
Code: 
gdb --args /usr/bin/pmtx status --changer msl2024

this opens an interactive gdb session. on the prompt,
add a breakpoint for 'rust_panic' with
Code: 
break rust_panic

after that let the command run with
Code: 
run
it then runs the given command
do that until a crash happens and gdb pauses on the panic breakpoint.
the do a
Code: 
thread apply all bt

to get a backtrace (on all threads)

second, we can try to debug the library behavior
can you install the 'mtx' package and do
Code: 
mtx -f <PATH_TO_CHANGER_DEVICE> status

if that results also in weird behaviour ?
 
  • Like
Reactions: flames
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back