Hello,
Some days ago, i set up proxmox-backup-client on my OMV VM to backup my photos and documents to my PBS server.
I have something like 350Go to backup (2 directory in the same batch).
My backup server is not really fast and behind a slow ethernet (100-150Mb/s).
So it's slow but ... i know why most of the time
My first backup took 7h39 to make and here is the summary :
So from here everything looked cool. It took around 350Go on the backup datastore. Ok seems fine.
Second day took 1h :
Perfect ! Just faster and no unneeded things transfered or space taken.
My last normal backup was this one (took 1h43) :
Seems ... really good
Didn't add/modify the documents/photos so 0 space on datastore because everything is duplicated from previous backup.
Perfect !
And this is the same (not always 100% duplicate but nearly) for some days.
At this time, proxmox-backup-client got killed every morning for OOM by OMV kernel for like 2 or 3 days (i reduced RAM allocation to OMV because i thought i gave it too much RAM .. well ... a mistake ).
Gave OMV more RAM and ran the backup again ... and here ... i think something is wrong.
Backup took 4h50 :
The deduplication ratio is really bad, upload size is enormous ... (and it took me 250Go approx on my datastore)
In 2 days I don't even know if i changed any document or photo. I think everything was the same the 08/03 and the 10/03 on my drives.
So here I need help to understand.
Why PBS did that ? Did i do something wrong? Because if I change nothing in the files and just stop the backup for 2 days and run it again ... duplication should stay at 100% in my mind.
If you have lights for me on this one ... I'll apreciate
Thanks
Some days ago, i set up proxmox-backup-client on my OMV VM to backup my photos and documents to my PBS server.
I have something like 350Go to backup (2 directory in the same batch).
My backup server is not really fast and behind a slow ethernet (100-150Mb/s).
So it's slow but ... i know why most of the time
My first backup took 7h39 to make and here is the summary :
Code:
2021-02-23T16:34:26+01:00: Upload statistics for 'photos.pxar.didx'
2021-02-23T16:34:26+01:00: UUID: 236b18a10b5e4e43bff32f28e8e1017e
2021-02-23T16:34:26+01:00: Checksum: c64a25eef5b2a21e0123f0f8c8b5950031a54216def3a6f677f6422b5dade377
2021-02-23T16:34:26+01:00: Size: 332049471286
2021-02-23T16:34:26+01:00: Chunk count: 87115
2021-02-23T16:34:26+01:00: Upload size: 300195113528 (90%)
2021-02-23T16:34:26+01:00: Duplicates: 9118+3586 (14%)
2021-02-23T16:34:26+01:00: Compression: 99%
2021-02-23T17:22:43+01:00: Upload statistics for 'virginie.pxar.didx'
2021-02-23T17:22:43+01:00: UUID: 2fbc8b3d2e8f4e3182faa83253fe0081
2021-02-23T17:22:43+01:00: Checksum: 7cff664860efb644096dd02490dbf4e50f0ad5bd673799da6b6ecf2342dc12b1
2021-02-23T17:22:43+01:00: Size: 35343914756
2021-02-23T17:22:43+01:00: Chunk count: 9377
2021-02-23T17:22:43+01:00: Upload size: 34954527391 (98%)
2021-02-23T17:22:43+01:00: Duplicates: 144+1040 (12%)
2021-02-23T17:22:43+01:00: Compression: 95%
Second day took 1h :
Code:
2021-02-24T01:52:37+01:00: Upload statistics for 'photos.pxar.didx'
2021-02-24T01:52:37+01:00: UUID: 9ec55587c735494a9410d3a72bd92388
2021-02-24T01:52:37+01:00: Checksum: ee1ebcd1ea88dbb2a3a1a3a8416dabb304ec45265769e847941e14f1bfe30afd
2021-02-24T01:52:37+01:00: Size: 332049471290
2021-02-24T01:52:37+01:00: Chunk count: 87115
2021-02-24T01:52:37+01:00: Upload size: 0 (0%)
2021-02-24T01:52:37+01:00: Duplicates: 87115+0 (100%)
2021-02-24T01:58:30+01:00: Upload statistics for 'virginie.pxar.didx'
2021-02-24T01:58:30+01:00: UUID: 6d883b04703d4873a5c04e127be83736
2021-02-24T01:58:30+01:00: Checksum: 7cff664860efb644096dd02490dbf4e50f0ad5bd673799da6b6ecf2342dc12b1
2021-02-24T01:58:30+01:00: Size: 35343914756
2021-02-24T01:58:30+01:00: Chunk count: 9377
2021-02-24T01:58:30+01:00: Upload size: 0 (0%)
2021-02-24T01:58:30+01:00: Duplicates: 9377+0 (100%)
My last normal backup was this one (took 1h43) :
Code:
2021-03-08T02:34:53+01:00: Upload statistics for 'photos.pxar.didx'
2021-03-08T02:34:53+01:00: UUID: 025ff7d9b26b4a43bc36696f33660210
2021-03-08T02:34:53+01:00: Checksum: ee1ebcd1ea88dbb2a3a1a3a8416dabb304ec45265769e847941e14f1bfe30afd
2021-03-08T02:34:53+01:00: Size: 332049471290
2021-03-08T02:34:53+01:00: Chunk count: 87115
2021-03-08T02:34:53+01:00: Upload size: 0 (0%)
2021-03-08T02:34:53+01:00: Duplicates: 87115+0 (100%)
2021-03-08T02:34:54+01:00: download 'virginie.pxar.didx' from previous backup.
2021-03-08T02:34:54+01:00: created new dynamic index 3 ("host/OMV/2021-03-08T00:00:01Z/virginie.pxar.didx")
2021-03-08T02:43:20+01:00: Upload statistics for 'virginie.pxar.didx'
2021-03-08T02:43:20+01:00: UUID: 282dbe66cc544bb7b2f0169e5e793828
2021-03-08T02:43:20+01:00: Checksum: 7cff664860efb644096dd02490dbf4e50f0ad5bd673799da6b6ecf2342dc12b1
2021-03-08T02:43:20+01:00: Size: 35343914756
2021-03-08T02:43:20+01:00: Chunk count: 9377
2021-03-08T02:43:20+01:00: Upload size: 0 (0%)
2021-03-08T02:43:20+01:00: Duplicates: 9377+0 (100%)
Didn't add/modify the documents/photos so 0 space on datastore because everything is duplicated from previous backup.
Perfect !
And this is the same (not always 100% duplicate but nearly) for some days.
At this time, proxmox-backup-client got killed every morning for OOM by OMV kernel for like 2 or 3 days (i reduced RAM allocation to OMV because i thought i gave it too much RAM .. well ... a mistake ).
Gave OMV more RAM and ran the backup again ... and here ... i think something is wrong.
Backup took 4h50 :
Code:
2021-03-10T21:17:37+01:00: Upload statistics for 'photos.pxar.didx'
2021-03-10T21:17:37+01:00: UUID: 3a2515f174614f2d9fa05cef0d207a7a
2021-03-10T21:17:37+01:00: Checksum: ef8dec6132d7ecd838f5b406102a3560597d20be41457995966a3398691566ca
2021-03-10T21:17:37+01:00: Size: 332049471290
2021-03-10T21:17:37+01:00: Chunk count: 87115
2021-03-10T21:17:37+01:00: Upload size: 215657613523 (64%)
2021-03-10T21:17:37+01:00: Duplicates: 40482+0 (46%)
2021-03-10T21:17:37+01:00: Compression: 98%
2021-03-10T21:17:37+01:00: created new dynamic index 3 ("host/OMV/2021-03-10T15:52:06Z/virginie.pxar.didx")
2021-03-10T21:42:58+01:00: Upload statistics for 'virginie.pxar.didx'
2021-03-10T21:42:58+01:00: UUID: e686039c92c44810be2a0fbccd26023a
2021-03-10T21:42:58+01:00: Checksum: 5207bb0d0603c774cfbaf3f118867dc4e948459dcbf1583683b6096f2b442108
2021-03-10T21:42:58+01:00: Size: 35343914756
2021-03-10T21:42:58+01:00: Chunk count: 9376
2021-03-10T21:42:58+01:00: Upload size: 21627468095 (61%)
2021-03-10T21:42:58+01:00: Duplicates: 4518+466 (53%)
2021-03-10T21:42:58+01:00: Compression: 95%
In 2 days I don't even know if i changed any document or photo. I think everything was the same the 08/03 and the 10/03 on my drives.
So here I need help to understand.
Why PBS did that ? Did i do something wrong? Because if I change nothing in the files and just stop the backup for 2 days and run it again ... duplication should stay at 100% in my mind.
If you have lights for me on this one ... I'll apreciate
Thanks
Last edited: