pbs performance vs restic

[kreutpet]

Hi,
just recently moved standard backup vzdump to the PBS (Backup Server 1.0-5).

First backup of my nextcloud lxc took 21min 41s

2020-12-05T23:30:10+01:00: starting new backup on datastore 'backupPVE': "ct/113/2020-12-05T22:30:03Z"
2020-12-05T23:30:10+01:00: add blob "/media/hd3/backupPVE/ct/113/2020-12-05T22:30:03Z/pct.conf.blob" (1139 bytes, comp: 1139)
2020-12-05T23:30:10+01:00: add blob "/media/hd3/backupPVE/ct/113/2020-12-05T22:30:03Z/fw.conf.blob" (86 bytes, comp: 86)
2020-12-05T23:30:10+01:00: created new dynamic index 1 ("ct/113/2020-12-05T22:30:03Z/catalog.pcat1.didx")
2020-12-05T23:30:10+01:00: created new dynamic index 2 ("ct/113/2020-12-05T22:30:03Z/root.pxar.didx")
2020-12-05T23:51:51+01:00: Upload statistics for 'root.pxar.didx'
2020-12-05T23:51:51+01:00: UUID: 4dabc166c71d413fbdae6d8b5b8d529d
2020-12-05T23:51:51+01:00: Checksum: 024c589be3eda500b6786d1b846185e5daced2b0aac5a60d3a8ffd120444affe
2020-12-05T23:51:51+01:00: Size: 45405190488
2020-12-05T23:51:51+01:00: Chunk count: 11815
2020-12-05T23:51:51+01:00: Upload size: 41206686977 (90%)
2020-12-05T23:51:51+01:00: Duplicates: 1200+333 (12%)
2020-12-05T23:51:51+01:00: Compression: 92%
2020-12-05T23:51:51+01:00: successfully closed dynamic index 2
2020-12-05T23:51:51+01:00: Upload statistics for 'catalog.pcat1.didx'
2020-12-05T23:51:51+01:00: UUID: 61590e65d97441d58168c9257ca0b7bd
2020-12-05T23:51:51+01:00: Checksum: 3c2c45e2fdd3b3f5f138f60641f7c75b76bc41fb84e27e883e2c882c3d300744
2020-12-05T23:51:51+01:00: Size: 3896368
2020-12-05T23:51:51+01:00: Chunk count: 11
2020-12-05T23:51:51+01:00: Upload size: 3896368 (100%)
2020-12-05T23:51:51+01:00: Compression: 36%
2020-12-05T23:51:51+01:00: successfully closed dynamic index 1
2020-12-05T23:51:51+01:00: add blob "/media/hd3/backupPVE/ct/113/2020-12-05T22:30:03Z/index.json.blob" (422 bytes, comp: 422)
2020-12-05T23:51:51+01:00: successfully finished backup
2020-12-05T23:51:51+01:00: backup finished successfully
2020-12-05T23:51:51+01:00: TASK OK

2nd : 19min 13s

2020-12-08T23:30:13+01:00: starting new backup on datastore 'backupPVE': "ct/113/2020-12-08T22:30:03Z"
2020-12-08T23:30:13+01:00: download 'index.json.blob' from previous backup.
2020-12-08T23:30:13+01:00: add blob "/media/hd3/backupPVE/ct/113/2020-12-08T22:30:03Z/pct.conf.blob" (1139 bytes, comp: 1139)
2020-12-08T23:30:13+01:00: add blob "/media/hd3/backupPVE/ct/113/2020-12-08T22:30:03Z/fw.conf.blob" (86 bytes, comp: 86)
2020-12-08T23:30:13+01:00: created new dynamic index 1 ("ct/113/2020-12-08T22:30:03Z/catalog.pcat1.didx")
2020-12-08T23:30:13+01:00: register chunks in 'root.pxar.didx' from previous backup.
2020-12-08T23:30:13+01:00: download 'root.pxar.didx' from previous backup.
2020-12-08T23:30:13+01:00: created new dynamic index 2 ("ct/113/2020-12-08T22:30:03Z/root.pxar.didx")
2020-12-08T23:49:26+01:00: Upload statistics for 'root.pxar.didx'
2020-12-08T23:49:26+01:00: UUID: cf38f458b8004a458eef2b3b8a9dfa1b
2020-12-08T23:49:26+01:00: Checksum: ca8127b5e4e828b273145058d672422d62fb0f5f81b73d98f0f351603dfda7b3
2020-12-08T23:49:26+01:00: Size: 45441440601
2020-12-08T23:49:26+01:00: Chunk count: 11802
2020-12-08T23:49:26+01:00: Upload size: 467098468 (1%)
2020-12-08T23:49:26+01:00: Duplicates: 11735+0 (99%)
2020-12-08T23:49:26+01:00: Compression: 25%
2020-12-08T23:49:26+01:00: successfully closed dynamic index 2
2020-12-08T23:49:26+01:00: Upload statistics for 'catalog.pcat1.didx'
2020-12-08T23:49:26+01:00: UUID: 4a68ceef3400461e95594645d2752bf2
2020-12-08T23:49:26+01:00: Checksum: 36a4574587f90f55b0d9614363100d59c4aff772741a7977345002b772b74409
2020-12-08T23:49:26+01:00: Size: 3897190
2020-12-08T23:49:26+01:00: Chunk count: 11
2020-12-08T23:49:26+01:00: Upload size: 3897190 (100%)
2020-12-08T23:49:26+01:00: Duplicates: 0+7 (63%)
2020-12-08T23:49:26+01:00: Compression: 36%
2020-12-08T23:49:26+01:00: successfully closed dynamic index 1
2020-12-08T23:49:26+01:00: add blob "/media/hd3/backupPVE/ct/113/2020-12-08T22:30:03Z/index.json.blob" (425 bytes, comp: 425)
2020-12-08T23:49:26+01:00: successfully finished backup
2020-12-08T23:49:26+01:00: backup finished successfully
2020-12-08T23:49:26+01:00: TASK OK

I expected that the speed of any new incremental update should be much fast.

i made a test of my restic backup of my laptop to a restic rest-server runing on the pve.

Creating incremental backup ...
open repository
repository 1f8d0284 opened successfully, password is correct
created new cache in /root/.cache/restic
lock repository
load index files
using parent snapshot a54fccee
start scan on [/]
start backup on [/]
scan finished in 13.747s: 342968 files, 15.975 GiB

Files: 62406 new, 13350 changed, 267212 unmodified
Dirs: 16082 new, 1248 changed, 27559 unmodified
Data Blobs: 20264 new
Tree Blobs: 17268 new
Added to the repo: 2.526 GiB

processed 342968 files, 15.975 GiB in 1:53
snapshot 806d4b26 saved

I was expecting that the backup based on pbs should be much faster compared to the restic.
restic so much fast even if the backup also needs be send via network to pve.
retic and pbs are saving to the same harddisk on the pbs/pve

any hint ?
thx

 

[dcsapak]

for now, pbs needs to read the data from all files of the source, but it is still deduplicated
restic AFAIK only backups files where the metadata changed (size, ctime/mtime?)
this mode is currently not possible on pbs, see discussions/explanations here:

https://bugzilla.proxmox.com/show_bug.cgi?id=3174
https://bugzilla.proxmox.com/show_bug.cgi?id=3138
https://lists.proxmox.com/pipermail/pve-devel/2020-November/045993.html

 

[tuxis]

Isn't a on-disk cache of 'filename,size,ctime,mtime,inode' the defacto standard for speeding up backups? It seems kinda obvious that reading all the data may be the coolest option, but really doesn't scale at all..

 

[kreutpet]

thanks for sharing. Things are more clear now that the technology can not really be compared.
As i am using the pvs/pbs for my private purpose i can actually life with it.

Not sure if the licenses of restic or borg would actually allow you to use them instead of reinventing it by yourself.
But actually that is the whole idea of opensource and you also to large extend use opensource.
So why not reconsidering this and use restic as platfrom and focus on the integration into your ecosystem

 

[dcsapak]

Isn't a on-disk cache of 'filename,size,ctime,mtime,inode' the defacto standard for speeding up backups? It seems kinda obvious that reading all the data may be the coolest option, but really doesn't scale at all..

there are always up/downsided to different approaches, but you can find our reasoning in the links i posted and e.g. here https://lists.proxmox.com/pipermail/pbs-devel/2020-December/001701.html

Not sure if the licenses of restic or borg would actually allow you to use them instead of reinventing it by yourself.
But actually that is the whole idea of opensource and you also to large extend use opensource.
So why not reconsidering this and use restic as platfrom and focus on the integration into your ecosystem

while restic uses some of the same principles we do (chunking/deduplication/etc..)
it does very different things, e.g. it does not concern itself with user management, vm backups, server side maintenance etc.