Hello,
I am currently restoring a VM from PBS. The VM disk is 5TB but there is only 1.1TB of data (thin provisioning). My PBS and my PVE are connected through VPN (both on a 1Gbps FTTH link). Data is usually sent at ~15Mbps. The initial backup took ~12 hours if I recall correctly. But the restore is very slow, here is the log:
(I stripped some lines in the log from 11% to 29% so this post doesn't reach the 15,000 characters limit).
It took 369358 seconds (102 hours) to reach 53% so I calculate it will take an additional ~90 hours for the remaining 47%.
I saw similar comments about VZdump but I don't know if they are relevant to PBS. Is there any method I can use to speed up the process? Like transferring the *.blob and *.fidx files manually to the PVE and restore from there maybe? If not, I'll just wait patiently...
Thanks for any help.
I am currently restoring a VM from PBS. The VM disk is 5TB but there is only 1.1TB of data (thin provisioning). My PBS and my PVE are connected through VPN (both on a 1Gbps FTTH link). Data is usually sent at ~15Mbps. The initial backup took ~12 hours if I recall correctly. But the restore is very slow, here is the log:
Code:
new volume ID is 'ssdpool-enc_type_zfs:vm-101-disk-0'
new volume ID is 'ssdpool-enc_type_zfs:vm-101-disk-1'
restore proxmox backup image: /usr/bin/pbs-restore --repository root@pam@192.168.14.12:backups vm/101/2020-11-14T21:22:54Z drive-scsi0.img.fidx /dev/zvol/ssdpool/enc_proxmox_directories/type_zfs/vm-101-disk-0 --verbose --format raw --skip-zero
connecting to repository 'root@pam@192.168.14.12:backups'
open block backend for target '/dev/zvol/ssdpool/enc_proxmox_directories/type_zfs/vm-101-disk-0'
starting to restore snapshot 'vm/101/2020-11-14T21:22:54Z'
download and verify backup index
progress 1% (read 432013312 bytes, zeroes = 35% (155189248 bytes), duration 69 sec)
progress 2% (read 859832320 bytes, zeroes = 51% (440401920 bytes), duration 107 sec)
progress 3% (read 1291845632 bytes, zeroes = 56% (729808896 bytes), duration 120 sec)
progress 4% (read 1719664640 bytes, zeroes = 42% (734003200 bytes), duration 171 sec)
progress 5% (read 2147483648 bytes, zeroes = 34% (738197504 bytes), duration 258 sec)
progress 6% (read 2579496960 bytes, zeroes = 33% (868220928 bytes), duration 301 sec)
progress 7% (read 3007315968 bytes, zeroes = 36% (1103101952 bytes), duration 332 sec)
progress 8% (read 3439329280 bytes, zeroes = 37% (1279262720 bytes), duration 359 sec)
progress 9% (read 3867148288 bytes, zeroes = 35% (1375731712 bytes), duration 398 sec)
progress 10% (read 4294967296 bytes, zeroes = 36% (1572864000 bytes), duration 425 sec)
progress 11% (read 4726980608 bytes, zeroes = 34% (1610612736 bytes), duration 487 sec)
progress 12% (read 5154799616 bytes, zeroes = 32% (1694498816 bytes), duration 534 sec)
progress 13% (read 5586812928 bytes, zeroes = 34% (1904214016 bytes), duration 557 sec)
progress 14% (read 6014631936 bytes, zeroes = 32% (1937768448 bytes), duration 598 sec)
progress 15% (read 6442450944 bytes, zeroes = 33% (2155872256 bytes), duration 624 sec)
progress 16% (read 6874464256 bytes, zeroes = 33% (2290089984 bytes), duration 659 sec)
progress 17% (read 7302283264 bytes, zeroes = 32% (2394947584 bytes), duration 686 sec)
progress 18% (read 7734296576 bytes, zeroes = 32% (2495610880 bytes), duration 723 sec)
progress 19% (read 8162115584 bytes, zeroes = 31% (2600468480 bytes), duration 753 sec)
progress 20% (read 8589934592 bytes, zeroes = 31% (2747269120 bytes), duration 771 sec)
progress 21% (read 9021947904 bytes, zeroes = 32% (2952790016 bytes), duration 786 sec)
progress 22% (read 9449766912 bytes, zeroes = 33% (3191865344 bytes), duration 813 sec)
progress 23% (read 9881780224 bytes, zeroes = 35% (3464495104 bytes), duration 822 sec)
progress 24% (read 10309599232 bytes, zeroes = 35% (3699376128 bytes), duration 836 sec)
progress 25% (read 10737418240 bytes, zeroes = 36% (3972005888 bytes), duration 841 sec)
progress 26% (read 11169431552 bytes, zeroes = 37% (4173332480 bytes), duration 874 sec)
progress 27% (read 11597250560 bytes, zeroes = 37% (4303355904 bytes), duration 915 sec)
progress 28% (read 12029263872 bytes, zeroes = 37% (4496293888 bytes), duration 935 sec)
progress 29% (read 12457082880 bytes, zeroes = 37% (4617928704 bytes), duration 960 sec)
progress 30% (read 12884901888 bytes, zeroes = 38% (4953473024 bytes), duration 964 sec)
progress 31% (read 13316915200 bytes, zeroes = 39% (5272240128 bytes), duration 968 sec)
progress 32% (read 13744734208 bytes, zeroes = 40% (5528092672 bytes), duration 978 sec)
progress 33% (read 14176747520 bytes, zeroes = 41% (5817499648 bytes), duration 987 sec)
progress 34% (read 14604566528 bytes, zeroes = 39% (5817499648 bytes), duration 1046 sec)
progress 35% (read 15032385536 bytes, zeroes = 40% (6157238272 bytes), duration 1049 sec)
progress 36% (read 15464398848 bytes, zeroes = 41% (6450839552 bytes), duration 1053 sec)
progress 37% (read 15892217856 bytes, zeroes = 42% (6731857920 bytes), duration 1059 sec)
progress 38% (read 16324231168 bytes, zeroes = 43% (7025459200 bytes), duration 1070 sec)
progress 39% (read 16752050176 bytes, zeroes = 42% (7151288320 bytes), duration 1105 sec)
progress 40% (read 17179869184 bytes, zeroes = 41% (7151288320 bytes), duration 1180 sec)
progress 41% (read 17611882496 bytes, zeroes = 41% (7344226304 bytes), duration 1227 sec)
progress 42% (read 18039701504 bytes, zeroes = 42% (7755268096 bytes), duration 1227 sec)
progress 43% (read 18471714816 bytes, zeroes = 44% (8128561152 bytes), duration 1230 sec)
progress 44% (read 18899533824 bytes, zeroes = 45% (8552185856 bytes), duration 1230 sec)
progress 45% (read 19327352832 bytes, zeroes = 46% (8967421952 bytes), duration 1230 sec)
progress 46% (read 19759366144 bytes, zeroes = 47% (9399435264 bytes), duration 1230 sec)
progress 47% (read 20187185152 bytes, zeroes = 48% (9827254272 bytes), duration 1230 sec)
progress 48% (read 20619198464 bytes, zeroes = 49% (10175381504 bytes), duration 1236 sec)
progress 49% (read 21047017472 bytes, zeroes = 50% (10565451776 bytes), duration 1239 sec)
progress 50% (read 21474836480 bytes, zeroes = 51% (10993270784 bytes), duration 1239 sec)
progress 51% (read 21906849792 bytes, zeroes = 52% (11425284096 bytes), duration 1239 sec)
progress 52% (read 22334668800 bytes, zeroes = 53% (11853103104 bytes), duration 1239 sec)
progress 53% (read 22766682112 bytes, zeroes = 53% (12180258816 bytes), duration 1250 sec)
progress 54% (read 23194501120 bytes, zeroes = 54% (12561940480 bytes), duration 1254 sec)
progress 55% (read 23622320128 bytes, zeroes = 54% (12989759488 bytes), duration 1254 sec)
progress 56% (read 24054333440 bytes, zeroes = 55% (13421772800 bytes), duration 1254 sec)
progress 57% (read 24482152448 bytes, zeroes = 56% (13849591808 bytes), duration 1254 sec)
progress 58% (read 24914165760 bytes, zeroes = 57% (14248050688 bytes), duration 1255 sec)
progress 59% (read 25341984768 bytes, zeroes = 57% (14675869696 bytes), duration 1255 sec)
progress 60% (read 25769803776 bytes, zeroes = 58% (15103688704 bytes), duration 1255 sec)
progress 61% (read 26201817088 bytes, zeroes = 59% (15535702016 bytes), duration 1255 sec)
progress 62% (read 26629636096 bytes, zeroes = 59% (15963521024 bytes), duration 1255 sec)
progress 63% (read 27061649408 bytes, zeroes = 60% (16290676736 bytes), duration 1266 sec)
progress 64% (read 27489468416 bytes, zeroes = 60% (16672358400 bytes), duration 1271 sec)
progress 65% (read 27917287424 bytes, zeroes = 61% (17100177408 bytes), duration 1271 sec)
progress 66% (read 28349300736 bytes, zeroes = 61% (17532190720 bytes), duration 1271 sec)
progress 67% (read 28777119744 bytes, zeroes = 62% (17960009728 bytes), duration 1271 sec)
progress 68% (read 29209133056 bytes, zeroes = 62% (18329108480 bytes), duration 1274 sec)
progress 69% (read 29636952064 bytes, zeroes = 63% (18756927488 bytes), duration 1274 sec)
progress 70% (read 30064771072 bytes, zeroes = 63% (19184746496 bytes), duration 1274 sec)
progress 71% (read 30496784384 bytes, zeroes = 64% (19616759808 bytes), duration 1274 sec)
progress 72% (read 30924603392 bytes, zeroes = 64% (20044578816 bytes), duration 1274 sec)
progress 73% (read 31356616704 bytes, zeroes = 65% (20401094656 bytes), duration 1280 sec)
progress 74% (read 31784435712 bytes, zeroes = 65% (20828913664 bytes), duration 1280 sec)
progress 75% (read 32212254720 bytes, zeroes = 65% (21256732672 bytes), duration 1280 sec)
progress 76% (read 32644268032 bytes, zeroes = 66% (21688745984 bytes), duration 1280 sec)
progress 77% (read 33072087040 bytes, zeroes = 66% (22116564992 bytes), duration 1280 sec)
progress 78% (read 33504100352 bytes, zeroes = 67% (22519218176 bytes), duration 1281 sec)
progress 79% (read 33931919360 bytes, zeroes = 67% (22942842880 bytes), duration 1281 sec)
progress 80% (read 34359738368 bytes, zeroes = 68% (23370661888 bytes), duration 1281 sec)
progress 81% (read 34791751680 bytes, zeroes = 68% (23802675200 bytes), duration 1281 sec)
progress 82% (read 35219570688 bytes, zeroes = 68% (24230494208 bytes), duration 1281 sec)
progress 83% (read 35651584000 bytes, zeroes = 69% (24645730304 bytes), duration 1281 sec)
progress 84% (read 36079403008 bytes, zeroes = 69% (25073549312 bytes), duration 1281 sec)
progress 85% (read 36507222016 bytes, zeroes = 69% (25501368320 bytes), duration 1281 sec)
progress 86% (read 36939235328 bytes, zeroes = 70% (25933381632 bytes), duration 1281 sec)
progress 87% (read 37367054336 bytes, zeroes = 70% (26361200640 bytes), duration 1281 sec)
progress 88% (read 37799067648 bytes, zeroes = 70% (26789019648 bytes), duration 1281 sec)
progress 89% (read 38226886656 bytes, zeroes = 71% (27216838656 bytes), duration 1281 sec)
progress 90% (read 38654705664 bytes, zeroes = 71% (27644657664 bytes), duration 1281 sec)
progress 91% (read 39086718976 bytes, zeroes = 71% (28076670976 bytes), duration 1281 sec)
progress 92% (read 39514537984 bytes, zeroes = 72% (28504489984 bytes), duration 1281 sec)
progress 93% (read 39946551296 bytes, zeroes = 72% (28932308992 bytes), duration 1281 sec)
progress 94% (read 40374370304 bytes, zeroes = 72% (29360128000 bytes), duration 1281 sec)
progress 95% (read 40802189312 bytes, zeroes = 73% (29787947008 bytes), duration 1281 sec)
progress 96% (read 41234202624 bytes, zeroes = 73% (30219960320 bytes), duration 1281 sec)
progress 97% (read 41662021632 bytes, zeroes = 73% (30647779328 bytes), duration 1281 sec)
progress 98% (read 42094034944 bytes, zeroes = 73% (31075598336 bytes), duration 1281 sec)
progress 99% (read 42521853952 bytes, zeroes = 74% (31503417344 bytes), duration 1281 sec)
progress 100% (read 42949672960 bytes, zeroes = 74% (31927042048 bytes), duration 1281 sec)
restore image complete (bytes=42949672960, duration=1281.75s, speed=31.96MB/s)
restore proxmox backup image: /usr/bin/pbs-restore --repository root@pam@192.168.14.12:backups vm/101/2020-11-14T21:22:54Z drive-scsi1.img.fidx /dev/zvol/ssdpool/enc_proxmox_directories/type_zfs/vm-101-disk-1 --verbose --format raw --skip-zero
connecting to repository 'root@pam@192.168.14.12:backups'
open block backend for target '/dev/zvol/ssdpool/enc_proxmox_directories/type_zfs/vm-101-disk-1'
starting to restore snapshot 'vm/101/2020-11-14T21:22:54Z'
download and verify backup index
progress 1% (read 53687091200 bytes, zeroes = 81% (43998248960 bytes), duration 2835 sec)
progress 2% (read 107374182400 bytes, zeroes = 72% (77506543616 bytes), duration 9793 sec)
progress 3% (read 161061273600 bytes, zeroes = 70% (112810000384 bytes), duration 16645 sec)
progress 4% (read 214748364800 bytes, zeroes = 69% (148704854016 bytes), duration 22991 sec)
progress 5% (read 268435456000 bytes, zeroes = 67% (181684666368 bytes), duration 30589 sec)
progress 6% (read 322122547200 bytes, zeroes = 67% (218519044096 bytes), duration 36536 sec)
progress 7% (read 375809638400 bytes, zeroes = 67% (255068209152 bytes), duration 42742 sec)
progress 8% (read 429496729600 bytes, zeroes = 67% (290250031104 bytes), duration 49534 sec)
progress 9% (read 483183820800 bytes, zeroes = 67% (328149762048 bytes), duration 55775 sec)
progress 10% (read 536870912000 bytes, zeroes = 67% (362715021312 bytes), duration 62730 sec)
progress 11% (read 590558003200 bytes, zeroes = 67% (398932836352 bytes), duration 69061 sec)
progress 29% (read 1556925644800 bytes, zeroes = 65% (1016535711744 bytes), duration 185562 sec)
progress 30% (read 1610612736000 bytes, zeroes = 64% (1046269132800 bytes), duration 192857 sec)
progress 31% (read 1664299827200 bytes, zeroes = 64% (1077936128000 bytes), duration 200680 sec)
progress 32% (read 1717986918400 bytes, zeroes = 64% (1115806498816 bytes), duration 205700 sec)
progress 33% (read 1771674009600 bytes, zeroes = 65% (1153957888000 bytes), duration 211568 sec)
progress 34% (read 1825361100800 bytes, zeroes = 65% (1191496908800 bytes), duration 217319 sec)
progress 35% (read 1879048192000 bytes, zeroes = 65% (1223281344512 bytes), duration 225584 sec)
progress 36% (read 1932735283200 bytes, zeroes = 64% (1253404835840 bytes), duration 234009 sec)
progress 37% (read 1986422374400 bytes, zeroes = 64% (1282110652416 bytes), duration 242249 sec)
progress 38% (read 2040109465600 bytes, zeroes = 64% (1310682251264 bytes), duration 251132 sec)
progress 39% (read 2093796556800 bytes, zeroes = 64% (1340327591936 bytes), duration 259610 sec)
progress 40% (read 2147483648000 bytes, zeroes = 63% (1368496537600 bytes), duration 269095 sec)
progress 41% (read 2201170739200 bytes, zeroes = 63% (1397336571904 bytes), duration 278366 sec)
progress 42% (read 2254857830400 bytes, zeroes = 63% (1428286341120 bytes), duration 286512 sec)
progress 43% (read 2308544921600 bytes, zeroes = 63% (1458602770432 bytes), duration 294719 sec)
progress 44% (read 2362232012800 bytes, zeroes = 63% (1489321852928 bytes), duration 303234 sec)
progress 45% (read 2415919104000 bytes, zeroes = 62% (1521089511424 bytes), duration 311108 sec)
progress 46% (read 2469606195200 bytes, zeroes = 62% (1553150771200 bytes), duration 319155 sec)
progress 47% (read 2523293286400 bytes, zeroes = 62% (1585576935424 bytes), duration 326956 sec)
progress 48% (read 2576980377600 bytes, zeroes = 62% (1619777290240 bytes), duration 333740 sec)
progress 49% (read 2630667468800 bytes, zeroes = 62% (1653767929856 bytes), duration 340520 sec)
progress 50% (read 2684354560000 bytes, zeroes = 62% (1685158100992 bytes), duration 348553 sec)
progress 51% (read 2738041651200 bytes, zeroes = 62% (1718691561472 bytes), duration 355537 sec)
progress 52% (read 2791728742400 bytes, zeroes = 62% (1753571393536 bytes), duration 362079 sec)
progress 53% (read 2845415833600 bytes, zeroes = 62% (1787377483776 bytes), duration 369358 sec)(I stripped some lines in the log from 11% to 29% so this post doesn't reach the 15,000 characters limit).
It took 369358 seconds (102 hours) to reach 53% so I calculate it will take an additional ~90 hours for the remaining 47%.
I saw similar comments about VZdump but I don't know if they are relevant to PBS. Is there any method I can use to speed up the process? Like transferring the *.blob and *.fidx files manually to the PVE and restore from there maybe? If not, I'll just wait patiently...
Thanks for any help.
Last edited:
