[SOLVED] Verification/skip-verified

showiproute

Well-Known Member
Mar 11, 2020
615
32
48
36
Austria
Hello everyone,

I am looking for a configuration which reverifies existing backups of VMs.
Currently I am running a verification monthly with "Skip verified" checked but also "Re-Verify after 30 days" enabled.

PBS is only verifying new backups and ignoring old ones - even if they have a flag that last verification is > 30 days.

Am I missunderstanding some of those config parameters?
 
What does your retention policy look like? Do you actually keep backups for longer than 30 days?
 
What does your retention policy look like? Do you actually keep backups for longer than 30 days?
Yes I do:

Code:
prune: s-953a925f-9901
        keep-daily 30
        keep-last 60
        keep-monthly 12
        keep-weekly 4
        keep-yearly 4
        schedule 2,22:30
        store Backup
 
Hello everyone,

I am looking for a configuration which reverifies existing backups of VMs.
Currently I am running a verification monthly with "Skip verified" checked but also "Re-Verify after 30 days" enabled.

PBS is only verifying new backups and ignoring old ones - even if they have a flag that last verification is > 30 days.

Am I missunderstanding some of those config parameters?
Hi,
please share the verification jobs task log. You can get it from the Administration > Task Panel of the PBS WebUI. Where the snapshots in question already older than 30 days at the time of the last scheduled verify job run?
 
Hi,
please share the verification jobs task log. You can get it from the Administration > Task Panel of the PBS WebUI. Where the snapshots in question already older than 30 days at the time of the last scheduled verify job run?
Hello @Chris ,

my verification job is currently still runing since 0000 hrs - I just copied some from the ongoing log:

Code:
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-31T18:00:03Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.28% (11/24 groups, 62/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-30T18:00:01Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.33% (11/24 groups, 63/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-29T18:00:00Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.39% (11/24 groups, 64/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-28T05:08:06Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.44% (11/24 groups, 65/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-27T20:28:16Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.50% (11/24 groups, 66/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-26T17:10:40Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.56% (11/24 groups, 67/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-25T17:07:38Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.61% (11/24 groups, 68/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-24T05:04:51Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.67% (11/24 groups, 69/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-23T17:09:07Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.72% (11/24 groups, 70/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-22T17:18:26Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.78% (11/24 groups, 71/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-21T05:04:17Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.83% (11/24 groups, 72/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-20T17:15:45Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.89% (11/24 groups, 73/75 snapshots in group #12)
2023-12-01T10:43:11+01:00: SKIPPED: verify Backup:vm/109/2023-10-19T17:05:50Z (recently verified)
2023-12-01T10:43:11+01:00: percentage done: 49.94% (11/24 groups, 74/75 snapshots in group #12)


As you might see from the backup date everything is > 30 days and from my point of view should be reverified.


Screenshot 2023-12-01 114434.png
 
Verfication config would be:

Code:
verification: v-49a10228-47a6
        ignore-verified true
        ns
        outdated-after 30
        schedule monthly
        store Backup
 
According to you task log, the backups where recently verified? Can you please check the last verification date for some of these backups by mouse hover over the yellow check-mark for the verify state?

Also, please provide your proxmox-backup-manager versions --verbose, so I can try to reproduce the issue.

Edit: Might there have been a date/time change of your system clock at some time which could lead to an incorrect verify date? Check your systemd journal and/or chrony service status.
 
Last edited:
According to you task log, the backups where recently verified? Can you please check the last verification date for some of these backups by mouse hover over the yellow check-mark for the verify state?

Also, please provide your proxmox-backup-manager versions --verbose, so I can try to reproduce the issue.

Edit: Might there have been a date/time change of your system clock at some time which could lead to an incorrect verify date? Check your systemd journal and/or chrony service status.
It says last verify date over 30 days ago but mentions 01-11-2023 00:00:00 - so that would be okay.

Code:
root@proxmox2:~# proxmox-backup-manager versions --verbose
proxmox-backup                     unknown      running kernel: 6.5.11-6-pve
proxmox-backup-server              3.1.2-1      running version: 3.1.2
proxmox-kernel-helper              8.1.0
pve-kernel-6.2                     8.0.5
proxmox-kernel-6.5                 6.5.11-6
proxmox-kernel-6.5.11-6-pve-signed 6.5.11-6
proxmox-kernel-6.5.11-5-pve-signed 6.5.11-5
proxmox-kernel-6.2.16-19-pve       6.2.16-19
proxmox-kernel-6.2                 6.2.16-19
pve-kernel-6.2.16-3-pve            6.2.16-3
ifupdown2                          3.2.0-1+pmx7
libjs-extjs                        7.0.0-4
proxmox-backup-docs                3.1.2-1
proxmox-backup-client              3.1.2-1
proxmox-mail-forward               0.2.2
proxmox-mini-journalreader         1.4.0
proxmox-offline-mirror-helper      0.6.3
proxmox-widget-toolkit             4.1.3
pve-xtermjs                        5.3.0-2
smartmontools                      7.3-pve1
zfsutils-linux                     2.2.0-pve4

So it seems that I missinterpreted the yellow check-mark.
 
It says last verify date over 30 days ago but mentions 01-11-2023 00:00:00 - so that would be okay.

Code:
root@proxmox2:~# proxmox-backup-manager versions --verbose
proxmox-backup                     unknown      running kernel: 6.5.11-6-pve
proxmox-backup-server              3.1.2-1      running version: 3.1.2
proxmox-kernel-helper              8.1.0
pve-kernel-6.2                     8.0.5
proxmox-kernel-6.5                 6.5.11-6
proxmox-kernel-6.5.11-6-pve-signed 6.5.11-6
proxmox-kernel-6.5.11-5-pve-signed 6.5.11-5
proxmox-kernel-6.2.16-19-pve       6.2.16-19
proxmox-kernel-6.2                 6.2.16-19
pve-kernel-6.2.16-3-pve            6.2.16-3
ifupdown2                          3.2.0-1+pmx7
libjs-extjs                        7.0.0-4
proxmox-backup-docs                3.1.2-1
proxmox-backup-client              3.1.2-1
proxmox-mail-forward               0.2.2
proxmox-mini-journalreader         1.4.0
proxmox-offline-mirror-helper      0.6.3
proxmox-widget-toolkit             4.1.3
pve-xtermjs                        5.3.0-2
smartmontools                      7.3-pve1
zfsutils-linux                     2.2.0-pve4

So it seems that I missinterpreted the yellow check-mark.
So I tried to reproduce, and for me it works as expected. The verify state of the backup is written as the last verify jobs date, so if the job is run each month, you might have slight variations as you sometimes have 30 days a month, other times 31 (or 28/29). You could increase the verify jobs frequency. The checkmark in the UI checks the verify state of the backup manifest against the current time, and warns if this is more than 30 days.
 
So I tried to reproduce, and for me it works as expected. The verify state of the backup is written as the last verify jobs date, so if the job is run each month, you might have slight variations as you sometimes have 30 days a month, other times 31 (or 28/29). You could increase the verify jobs frequency. The checkmark in the UI checks the verify state of the backup manifest against the current time, and warns if this is more than 30 days.
I guess that's my issue: Verification is done each month but as counts every second a verification might be already "yellow" if the verification was done 30 days +1 second ago.
 
  • Like
Reactions: Chris

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!