[SOLVED] PBS shows more then 500GiB used but summary of backuped data is less than 200
- Thread starter muekno
- Start date
Hey,
GC marks unused chucks which are removed ~1 day later, so they have to be marked as unused for some time before removal. For a technical overview you can take a look at [1].
[1] https://pbs.proxmox.com/docs/maintenance.html#gc-background
GC marks unused chucks which are removed ~1 day later, so they have to be marked as unused for some time before removal. For a technical overview you can take a look at [1].
[1] https://pbs.proxmox.com/docs/maintenance.html#gc-background
Not really, chunks basically have to be "unused" for ~1day.
Technically you could set your system time to a few days in the future and run GC, but that is NOT recommended.
Technically you could set your system time to a few days in the future and run GC, but that is NOT recommended.
Last edited:
My situation is, I have some VM's , with 2 x 16 and 2 x 32GiB and one with 50GIB and another with 600GiB.
My Backup Storage is 1TB, same as my VM space on PVE. From the smaler one I just like to hold one old backup, so I set retention to on.
From the 600GiB one I do not need to hold an old backup, so I set retention to 0, but I like to have a daily backup, so I scheduled it for every day.
If I understand you right that would not work. Can I solve it if I do a backup only every second or third day, and how to schedule it, as I saw, there is only daily, weekly, monthly, and yearly possible. Only the 50GiB is scheduled daily too, the other ones week and monthly, with them I will not have a problem as I see
Regards
Rainer
My Backup Storage is 1TB, same as my VM space on PVE. From the smaler one I just like to hold one old backup, so I set retention to on.
From the 600GiB one I do not need to hold an old backup, so I set retention to 0, but I like to have a daily backup, so I scheduled it for every day.
If I understand you right that would not work. Can I solve it if I do a backup only every second or third day, and how to schedule it, as I saw, there is only daily, weekly, monthly, and yearly possible. Only the 50GiB is scheduled daily too, the other ones week and monthly, with them I will not have a problem as I see
Regards
Rainer
Are the 450 GB from previous backups of these VMs? Most of the data probably has not changed. So you would delete this data and add it again.
IMO you try to solve a problem which does not exist. If you look at https://pbs.proxmox.com/docs/technical-overview.html#fixed-sized-chunks, you see, that a VM backup is broken into chunks of 4MiB. Each chunk is identified by the SHA-256 checksum of its content. The same content will be hashed to the same checksum. Each chunk with the same checksum (read content) is stored only once!
A Snapshot (or one Backup) of a VM consists of manifest, blobs and index files. The index files store which chunks belong to this Snapshot. Have a look at: https://pbs.proxmox.com/docs/technical-overview.html#snapshots. If a chunk is already contained in a previous snapshot on the PBS, then only the checksum is send.
A classical backup program with the Grandfather-Father-Son principle would store a full backup and incremental ones. If you delete there a full backup, the incremental ones depending on it, will also be destroyed. Two full backups in the classical approach need twice the disk space of the original data till some kind of deduplication takes place.
PBS is using a different approach than classical backup programs. A snapshot in PBS stores references to the data chunks. As explained above, a chunk with the same data is stored only once. The deduplication is taking place on the fly. Two full backups of the same VM take approximately the same space than a full and an incremental one in a classical backup program. And the second snapshot is still complete, if you delete the first snapshot.
In my case a VM which uses 1.52TB on the PVE is using less than 1.59TB for 11 days of backup on the PBS. I have only 11 days, because then I recently added more disk space to the PBS to backup this VM.
Best regards,
Karl-Heinz
IMO you try to solve a problem which does not exist. If you look at https://pbs.proxmox.com/docs/technical-overview.html#fixed-sized-chunks, you see, that a VM backup is broken into chunks of 4MiB. Each chunk is identified by the SHA-256 checksum of its content. The same content will be hashed to the same checksum. Each chunk with the same checksum (read content) is stored only once!
A Snapshot (or one Backup) of a VM consists of manifest, blobs and index files. The index files store which chunks belong to this Snapshot. Have a look at: https://pbs.proxmox.com/docs/technical-overview.html#snapshots. If a chunk is already contained in a previous snapshot on the PBS, then only the checksum is send.
A classical backup program with the Grandfather-Father-Son principle would store a full backup and incremental ones. If you delete there a full backup, the incremental ones depending on it, will also be destroyed. Two full backups in the classical approach need twice the disk space of the original data till some kind of deduplication takes place.
PBS is using a different approach than classical backup programs. A snapshot in PBS stores references to the data chunks. As explained above, a chunk with the same data is stored only once. The deduplication is taking place on the fly. Two full backups of the same VM take approximately the same space than a full and an incremental one in a classical backup program. And the second snapshot is still complete, if you delete the first snapshot.
In my case a VM which uses 1.52TB on the PVE is using less than 1.59TB for 11 days of backup on the PBS. I have only 11 days, because then I recently added more disk space to the PBS to backup this VM.
Best regards,
Karl-Heinz
Last edited:
