PBS tasks logs

[antipiot]

Hey!

Is there anyway to retreive tasks logs in command line?

I used to use "pvesh" command to get the logs of vzdump to check on backup status: is there anything close on PBS?

Thanks!

 

[dcsapak]

while there is no tool that is as flexible as 'pvesh' (yet),
proxmox-backup-manager can do what you want i believe:

proxmox-backup-manager task list
proxmox-backup-manager task log <UPID>

 

[antipiot]

Thanks for your answer!
This is exactly what i was looking for, i have adapted my script.



Is there a place where i could find all the error that may be reported in thoses logs?

 

[antipiot]

Is there anywhere i can find the full log for a specific task?
As prune, sync garbage collection?

 

[oversite]

Is there anywhere i can find the full log for a specific task?
As prune, sync garbage collection?

I find all task logging I need with the GUI upper right, klick tasks, then all tasks, the task list shows, double click the one you need.

 

[antipiot]

I find all task logging I need with the GUI upper right, klick tasks, then all tasks, the task list shows, double click the one you need.

Thanks for your answer, my question was not accurate

I'm looking to get thoses logs using CLI and would like to find if there are separate logs files for the different tasks.

 

[oversite]

Thanks for your answer, my question was not accurate

I'm looking to get thoses logs using CLI and would like to find if there are separate logs files for the different tasks.

Possibly you want these: /var/log/proxmox-backup/tasks.
You can ofcourse get them with: proxmox-backup-manager task list --all
and proxmox-backup-manager task log UPID

 

[antipiot]

sorry that i don't know. I don't know where the logs are.
You can ofcourse get them with: proxmox-backup-manager task list --all
and proxmox-backup-manager task log UPID

I do so, but it only output the last 50 lines mixing any tasks:


If you want to check the history of prune jobs that runs once in a week, there's likely no chance to find more than one entry in this logs.

 

[oversite]

I do so, but it only output the last 50 lines mixing any tasks:
View attachment 19003

If you want to check the history of prune jobs that runs once in a week, there's likely no chance to find more than one entry in this logs.

I see, then you want these: /var/log/proxmox-backup/tasks, there i find old logs of the tasks

 

[antipiot]

I see, then you want these: /var/log/proxmox-backup/tasks, there i find old logs of the tasks

Thanks, i hoped for a command line to output thoses but hey.

Adapted my script to recures into this directory to check for errors, looks to be working fine ATM

Regards - JS

 

[santiagobiali]

Had to parse some pbs sync logs today and this was the only way I found to get what I needed:

for i in $(grep "TASK OK" /var/log/proxmox-backup/tasks/*/*syncjob* | sed -e "s/root@pam:.*/root@pam:/g"); do start=$(grep -E "Starting" $i | sed -e "s/: Starting datastore sync job /,/g" -e "s/::.*//g"); end=$(grep -Eo ".*TASK OK" $i | sed -e "s/: TASK OK//g"); echo $(hostname),${start},${end}; done | sed -e "s/'//g" | column --table -s ',' --table-columns 'Host,StartDate,DataStore,EndDate'
Host  StartDate                  DataStore                     EndDate
b04   2023-05-16T09:00:36-03:00  cb02:archive1-pbs:orico1-pbs  2023-05-16T15:36:33-03:00
b04   2023-05-16T16:00:00-03:00  cb02:archive1-pbs:orico1-pbs  2023-05-16T16:00:11-03:00
b04   2023-05-16T17:00:00-03:00  cb02:archive1-pbs:orico1-pbs  2023-05-16T17:00:11-03:00

 

[antipiot]

Had to parse some pbs sync logs today and this was the only way I found to get what I needed:

for i in $(grep "TASK OK" /var/log/proxmox-backup/tasks/*/*syncjob* | sed -e "s/root@pam:.*/root@pam:/g"); do start=$(grep -E "Starting" $i | sed -e "s/: Starting datastore sync job /,/g" -e "s/::.*//g"); end=$(grep -Eo ".*TASK OK" $i | sed -e "s/: TASK OK//g"); echo $(hostname),${start},${end}; done | sed -e "s/'//g" | column --table -s ',' --table-columns 'Host,StartDate,DataStore,EndDate'
Host  StartDate                  DataStore                     EndDate
b04   2023-05-16T09:00:36-03:00  cb02:archive1-pbs:orico1-pbs  2023-05-16T15:36:33-03:00
b04   2023-05-16T16:00:00-03:00  cb02:archive1-pbs:orico1-pbs  2023-05-16T16:00:11-03:00
b04   2023-05-16T17:00:00-03:00  cb02:archive1-pbs:orico1-pbs  2023-05-16T17:00:11-03:00

Very clean way to do that!
I wanted to do something like this without taking the time. Thanks you for sharing!

 

[santiagobiali]

Just revisited this code, here are some improvements like showing how much data was transfered (empty if none).

Proxmox Sync Log Parser:

find /var/log/proxmox-backup/tasks/*/*syncjob* -maxdepth 0 -type f -mtime -30 -print | while read -r file; do
    if [[ -n "$(grep "TASK OK" "$file")" ]]; then
        start=$(grep -E "Starting" $file | sed -e "s/: Starting datastore sync job /,/g" -e "s/::.*//g")
        end=$(grep -Eo ".*TASK OK" $file | sed -e "s/: TASK OK//g")
        downloaded=$(grep -Eio "downloaded.*(B|bytes) " $file | sed -e "s/downloaded //g" -e "s/ TiB/*1073741824/g" -e "s/ GiB/*1048576/g" -e "s/ MiB/*1024/g" -e "s/ KiB/*1024/g" -e "s/ B//g" -e "s/ bytes//g" | sed ":a;$!N;s/\n/+/I;ta;" | bc)
        echo $(hostname),${start},${end},${downloaded}
    fi
done | column --table -s ',' --table-columns 'Host,StartDate,DataStore,EndDate,Transferred'

Host  StartDate                  DataStore                                 EndDate                    Transferred
b04   2024-08-14T09:00:00-03:00  'cb02:poolb3-pbs:orico1-pbs2              2024-08-14T09:10:22-03:00  10186249.216
b04   2024-08-14T13:00:00-03:00  'cb02:poolb3-pbs:orico1-pbs2              2024-08-14T13:11:22-03:00  28233419.776
b04   2024-08-12T08:00:00-03:00  'cb02:poolb3-pbs:orico1-pbs2              2024-08-12T17:17:56-03:00  1048651646.976