Here's a quick and dirty script, which parses and analyzes the proxmox VE dump log files on a node. It prints id, name and type of the object beeing backed up, the data transferred, if a dirtymap existed, the average speed of the transfer and the duration of the backup of container/vm and the duration of the whole job. It scans for all log files under /var/log/pve/tasks and processes them all. So it helps to keep track of the backup times of individual containers, virtual machines and backup jobs in whole.
Example a) Full program output
The script is available here:
https://codeberg.org/megabert/proxmox-ve-scripts/src/branch/master
It needs gawk to be installed.
Example b) Track backup times of a single VM
Example c) Track backup job times
Example a) Full program output
Code:
# vzdump-analyze
File-ID Date Time Obj-ID Obj-Type Obj-Name DirtyMap Tx-Size Speed Duration
6674D091 2024-06-21 03:00:01 100 VM opnsense 1 1.85 GiB 25.2 MiB/s 80
6674D091 2024-06-21 03:00:01 101 VM template 0 32.00 GiB 368.2 MiB/s 93
6674D091 2024-06-21 03:00:01 102 CT CT102 - 20.653 MiB 3.599 MiB/s 8
6674D091 2024-06-21 03:00:01 103 CT template - 49.44 MiB 8.826 MiB/s 7
6674D091 2024-06-21 03:00:01 107 VM Postgres-alt 1 140.00 MiB 7.8 MiB/s 21
6674D091 2024-06-21 03:00:01 111 CT backup - 32.644 MiB 5.336 MiB/s 8
6674D091 2024-06-21 03:00:01 161 VM test-cryptpad 0 32.00 GiB 885.6 MiB/s 40
6674D091 2024-06-21 03:00:01 1021 CT crowdsec-db - 697.971 MiB 50.808 MiB/s 18
6674D091 2024-06-21 03:00:01 2011 CT auth - 82.511 MiB 3.946 MiB/s 30
6674D091 2024-06-21 03:00:01 8102 CT wiki-updatetest - 192.966 MiB 4.347 MiB/s 47
6674D091 2024-06-21 03:00:01 8111 CT test-taiga - 0 B 0 B/s 31
6674D091 2024-06-21 03:00:01 8131 VM test-leantime 0 32.00 GiB 885.6 MiB/s 40
6674D091 2024-06-21 03:00:01 8141 VM test-matrix 0 32.00 GiB 799.2 MiB/s 44
6674D091 2024-06-21 03:00:01 8151 VM test-forum 0 32.00 GiB 409.6 MiB/s 83
...
6674D091 2024-06-21 03:00:01 backup job run duration 00:48:26
...
File-ID Date Time Obj-ID Obj-Type Obj-Name DirtyMap Tx-Size Speed Duration
6675CF01 2024-06-21 21:05:37 102 CT CT102 - 18.992 MiB 3.379 MiB/s 7
6675CF01 2024-06-21 21:05:37 backup job run duration 00:00:07
Explanations:
- File-ID: last 8 Chars of Identifier within the log file name, so you can find it again
- Date/Time: Start Time of Backup Job
- DirtyMap: VM-Only: Was there an existing Dirtymap used, or had it to be created new? (existing => fast incremental Backup)
- Tx-Size: Transferred Size of the Backup
- Duration: Backup Duration in SecondsThe script is available here:
https://codeberg.org/megabert/proxmox-ve-scripts/src/branch/master
It needs gawk to be installed.
Example b) Track backup times of a single VM
Code:
# vzdump-analyze |grep pmg01
File-ID Date Time Obj-ID Obj-Type Obj-Name DirtyMap Tx-Size Speed Duration
66575E72 2024-05-29 18:57:22 149 VM pmg01.mgt.mydomain.de 0 60.00 GiB 18.3 MiB/s 3370
66588492 2024-05-30 15:52:18 149 VM pmg01.mgt.mydomain.de 1 5.73 GiB 21.3 MiB/s 282
6658B3D0 2024-05-30 19:13:53 149 VM pmg01.mgt.mydomain.de 1 1.11 GiB 9.2 MiB/s 129
6658F5AE 2024-05-30 23:54:54 149 VM pmg01.mgt.mydomain.de 1 2.63 GiB 7.1 MiB/s 384
666FFE78 2024-06-17 11:14:32 149 VM pmg01.mgt.mydomain.de 1 21.69 GiB 11.2 MiB/s 1986
667030CF 2024-06-17 14:49:19 149 VM pmg01.mgt.mydomain.de 1 1.48 GiB 8.3 MiB/s 188
66705411 2024-06-17 17:19:45 149 VM pmg01.mgt.mydomain.de 1 1.19 GiB 7.1 MiB/s 178
6671E880 2024-06-18 22:05:21 149 VM pmg01.mgt.mydomain.de 1 7.13 GiB 6.7 MiB/s 1101
66720604 2024-06-19 00:11:16 149 VM pmg01.mgt.mydomain.de 1 2.74 GiB 7.4 MiB/s 384
667362F7 2024-06-20 01:00:07 149 VM pmg01.mgt.mydomain.de 1 6.58 GiB 18.1 MiB/s 379
6674B472 2024-06-21 01:00:02 149 VM pmg01.mgt.mydomain.de 1 7.52 GiB 12.4 MiB/s 629
667605F4 2024-06-22 01:00:05 149 VM pmg01.mgt.mydomain.de 1 7.24 GiB 8.6 MiB/s 881
66775771 2024-06-23 01:00:01 149 VM pmg01.mgt.mydomain.de 1 6.29 GiB 10.6 MiB/s 610Example c) Track backup job times
Code:
# vzdump-analyze | grep job
6599EC12 backup job not finished correctly
65A9A093 2024-01-18 23:05:07 backup job run duration 00:18:16
65DD0899 backup job not finished correctly
66574939 backup job not finished correctly
66574971 backup job not finished correctly
66575E72 2024-05-29 18:57:22 backup job run duration 09:09:23
66588492 2024-05-30 15:52:18 backup job run duration 01:31:24
6658B3D0 2024-05-30 19:13:53 backup job run duration 00:23:50
6658F5AE 2024-05-30 23:54:54 backup job run duration 00:59:50
666FFE78 2024-06-17 11:14:32 backup job run duration 03:34:14
667030CF 2024-06-17 14:49:19 backup job run duration 00:49:37
66705411 2024-06-17 17:19:45 backup job run duration 00:39:33
6671E880 2024-06-18 22:05:21 backup job run duration 02:02:25
66720604 2024-06-19 00:11:16 backup job run duration 01:33:54
667362F7 2024-06-20 01:00:07 backup job run duration 01:10:33
6674B472 2024-06-21 01:00:02 backup job run duration 01:23:00
667605F4 2024-06-22 01:00:05 backup job run duration 01:27:48
66775771 2024-06-23 01:00:01 backup job run duration 01:54:42
Last edited:
