I have 2 PBS servers, with the secondary intended to be a 1:1 mirror of the primary. What I'm noticing is that the primary server's prune jobs and garbage collection seem to be functioning properly, but the secondary's isn't. Nothing is backed up directly to the secondary server, it's only used as a sync target of the primary (off site, but in a nearby office). Both servers have a 10TB datastore. The primary is only using a little over 5TB, but the secondary is maxed out. How can I make these servers sync perfectly 1:1. Manually running prune & GC jobs don't seem to be freeing up much - I was able to free up a couple GB, but it quickly ran out again. Thanks!
PBS sync not 1:1?
- Thread starter whill1219
- Start date
Is the retention (keep) settings the same on both?
So this is a push job? We use a pull job and then the retention is set with a prune job on PBS2.
Do you have multiple namespaces or something? You can browse backups on the Content tab.
So this is a push job? We use a pull job and then the retention is set with a prune job on PBS2.
Do you have multiple namespaces or something? You can browse backups on the Content tab.
GC job must be set on each PBS.
Prune should be set on each PBS,
Snapshot sync deletions called "Remove vanished" isn't safe from my point of view.
See : https://forum.proxmox.com/threads/g...ary-if-remove-vanished-is-checked-eom.158266/
Prune should be set on each PBS,
Snapshot sync deletions called "Remove vanished" isn't safe from my point of view.
See : https://forum.proxmox.com/threads/g...ary-if-remove-vanished-is-checked-eom.158266/
I agree that enabling "Remove vanished" is a bad idea.
Imagine this scenario: We have one local (pbs-local) and one remote PBS (pbs-remote). They are configured like this:
- pbs-remote can only be accessed via vpn and it's iptables firewall is configured that it can only be accessed via vpn and normally only from your management client (aka the vpn ip of your notebook, workstation or jumpserver)
- The local ProxmoxVE nodes are allowed to create backups on pbs-local and restore them, but not not remove or otherwise alter them
- pbs-local has port 8007 opened in it's firewall that the vpn ips of pbs-remote and your managment client can access port 8007. The access permissions allow local clients (as already mentioned) to create and restore backups. They also allow pbs-remote to pull backups from pbs-local
- In case of a restore you would create temporary firewall rule to allow connections from pbs-local or your local pve-nodes to pbs-remote and setup permissions, that they can restore the backups from pbs-remote. After restore you would again remove any access (in the firewall as well as in PBS permissions)
Normally such a setup would prevent any ransomware attacks from your PVE nodes or from pbs-local since pbs-remote has it's ports closed as long your management client isn't taken over or the attackers don't manage to get around the firewall of pbs-remote. If attackers can't access pbs-remote how should they be able to do anything with them?
Now assume somehow a hacker or ransomware virus somehow gets access to pbs-local and remove your backups. Now what will happen to your offsite backup (pbs-remote)?
- You ticked "remote-vanished"->Your remote backups will be gone too after the next sync. This would even work if you implemented permissions and firewall rules as described above.
- You didn't ticked "remote-vanished"->You can still reinstall your local infrastructure and restore your stuff from the pbs-remote
For this reason I would recommend to use your remote PBS do keep older copies than your local data (think cold-storage) and to NOT tick "remote-vanished" at all. I would also recommend to setup your infrastructure as described above so your equivalent to pbs-remote can not be accessed from your local infrastructure.
See also https://pbs.proxmox.com/docs/storage.html#ransomware-protection-recovery https://pbs.proxmox.com/docs/user-management.html and a lot of earlier discussions in this forum how to protect against ransomware.
Imho "remote-vanished" should never been implemented but I can easily imagine a situation how an important customer demanded it so it was done, after warning them that it's a stupid idea
Imagine this scenario: We have one local (pbs-local) and one remote PBS (pbs-remote). They are configured like this:
- pbs-remote can only be accessed via vpn and it's iptables firewall is configured that it can only be accessed via vpn and normally only from your management client (aka the vpn ip of your notebook, workstation or jumpserver)
- The local ProxmoxVE nodes are allowed to create backups on pbs-local and restore them, but not not remove or otherwise alter them
- pbs-local has port 8007 opened in it's firewall that the vpn ips of pbs-remote and your managment client can access port 8007. The access permissions allow local clients (as already mentioned) to create and restore backups. They also allow pbs-remote to pull backups from pbs-local
- In case of a restore you would create temporary firewall rule to allow connections from pbs-local or your local pve-nodes to pbs-remote and setup permissions, that they can restore the backups from pbs-remote. After restore you would again remove any access (in the firewall as well as in PBS permissions)
Normally such a setup would prevent any ransomware attacks from your PVE nodes or from pbs-local since pbs-remote has it's ports closed as long your management client isn't taken over or the attackers don't manage to get around the firewall of pbs-remote. If attackers can't access pbs-remote how should they be able to do anything with them?
Now assume somehow a hacker or ransomware virus somehow gets access to pbs-local and remove your backups. Now what will happen to your offsite backup (pbs-remote)?
- You ticked "remote-vanished"->Your remote backups will be gone too after the next sync. This would even work if you implemented permissions and firewall rules as described above.
- You didn't ticked "remote-vanished"->You can still reinstall your local infrastructure and restore your stuff from the pbs-remote
For this reason I would recommend to use your remote PBS do keep older copies than your local data (think cold-storage) and to NOT tick "remote-vanished" at all. I would also recommend to setup your infrastructure as described above so your equivalent to pbs-remote can not be accessed from your local infrastructure.
See also https://pbs.proxmox.com/docs/storage.html#ransomware-protection-recovery https://pbs.proxmox.com/docs/user-management.html and a lot of earlier discussions in this forum how to protect against ransomware.
Imho "remote-vanished" should never been implemented but I can easily imagine a situation how an important customer demanded it so it was done, after warning them that it's a stupid idea