Tape Backup Task Output Limit? CLI Means of Viewing?

[BriBriBri]

Hi All,

Is there some CLI or alternate means of viewing the task output of a running tape backup task?

I ask because we are currently in the midst of a 25-30 TB backup to LTO-8 tape (on a Quantum SuperLoader 3, HH drive) and at about 20 hours into the task, the output window stopped updating while the task continues to run. I suspect this is a "display issue" rather than an issue with the task itself because there is one final line that pops in and out at the bottom of the output window (which is odd behavior). But it never adds anything new beyond that one oddly-behaving line.

2022-01-21T10:15:21-08:00: wrote 2468 chunks (4295.23 MB at 86.16 MB/s)
2022-01-21T10:16:08-08:00: wrote 2294 chunks (4295.49 MB at 90.48 MB/s)
2022-01-21T10:16:57-08:00: wrote 2255 chunks (4295.75 MB at 90.64 MB/s)
2022-01-21T10:17:09-08:00: wrote 640 chunks (1071.64 MB at 119.77 MB/s)
2022-01-21T10:17:16-08:00: end backup pbs1-primary:vm/163/2022-01-20T10:00:02Z
2022-01-21T10:17:16-08:00: percentage done: 34.41% (64/186 groups)
2022-01-21T10:17:17-08:00: backup snapshot vm/164/2022-01-20T02:00:02Z

I'm hoping the backup task completes over the next day or three. But I'd certainly like some visibility into what's going on. We're remote from the PBS server and Superloader 3 so getting eyes on either is not practical at the moment. IPMI on the SuperLoader 3 (recently replaced) not yet up. Any advice?

Thanks!

Brian

 

[BriBriBri]

Nevermind! Apparently, when PBS begins the process of writing a larger snapshot to tape (in this case, about 10TB), it takes some time to gather itself together before writing. I went back and noted this same behavior overnight when it wrote another 10TB snapshot. But once it's done with whatever preparation/processing it's doing, it continues on and the task output is updated normally.

 

[dietmar]

You can use the command line tools to list/view the task logs (on the backup server), see:

# proxmox-backup-manager help task

Also, may I ask what kind of backup storage do you use? Your tape backup speed seems a bit too low (should be above 150 MB/s).

 

[BriBriBri]

Hi! Thank you for the info about proxmox-backup-manager. Regarding our backup speed, it is generally hovering around 150 MB/s. Dipping a bit at times and sometimes peaking as high as 250 MB/s for a few task log entries. Here's a snippet chosen at random from our (now finished) backup job.

2022-01-20T14:55:56-08:00: wrote 1027 chunks (4295.23 MB at 179.93 MB/s)
2022-01-20T14:56:25-08:00: wrote 1029 chunks (4295.75 MB at 151.18 MB/s)
2022-01-20T14:56:55-08:00: wrote 1026 chunks (4296.02 MB at 160.61 MB/s)
2022-01-20T14:57:19-08:00: wrote 1026 chunks (4296.54 MB at 177.96 MB/s)
2022-01-20T14:57:44-08:00: wrote 1026 chunks (4296.28 MB at 179.62 MB/s)
2022-01-20T14:58:11-08:00: wrote 1027 chunks (4298.11 MB at 175.55 MB/s)
2022-01-20T14:58:38-08:00: wrote 1028 chunks (4298.90 MB at 162.50 MB/s)
2022-01-20T14:59:05-08:00: wrote 1027 chunks (4298.38 MB at 163.47 MB/s)
2022-01-20T14:59:33-08:00: wrote 1026 chunks (4298.90 MB at 165.36 MB/s)
2022-01-20T14:59:59-08:00: wrote 1028 chunks (4296.54 MB at 179.55 MB/s)
2022-01-20T15:00:27-08:00: wrote 1026 chunks (4297.33 MB at 171.14 MB/s)
2022-01-20T15:00:54-08:00: wrote 1026 chunks (4297.85 MB at 173.17 MB/s)
2022-01-20T15:01:23-08:00: wrote 1027 chunks (4297.06 MB at 172.01 MB/s)
2022-01-20T15:01:53-08:00: wrote 1027 chunks (4297.85 MB at 152.44 MB/s)
2022-01-20T15:02:20-08:00: wrote 1027 chunks (4297.06 MB at 180.47 MB/s)
2022-01-20T15:02:51-08:00: wrote 1027 chunks (4298.64 MB at 151.10 MB/s)
2022-01-20T15:03:19-08:00: wrote 1027 chunks (4298.64 MB at 160.95 MB/s)
2022-01-20T15:03:50-08:00: wrote 1027 chunks (4297.59 MB at 159.36 MB/s)
2022-01-20T15:04:17-08:00: wrote 1027 chunks (4296.54 MB at 188.55 MB/s)
2022-01-20T15:04:45-08:00: wrote 1026 chunks (4296.80 MB at 161.40 MB/s)

While I wish it didn't, our pbs server consists of 24 SATA HDDs (not SSDs). Model of drives is ST10000NM0086-2AA101.

It's sufficient for our purposes. But of course, come budget season, I'd love to replace with SSDs, or perhaps retain this server for longer-term backup storage (via PBS sync job?) and tape writing and build a separate server with SSDs for our nightly backups.

Thanks again!

--Brian