initial VM backup

[damon1]

just a thought.

could you upload via local media a copy of the initial VM.
Similar to CLONE but CLONE to USB
then take the USB to the BACKUP machine and UPLOAD the initial VM via UPLOAD etc... .

On the local network it took about 10 minutes on 10GB LAN but WAN, I expect it to take a quite a few hours.

These are the first 2 VM's which are 48GB Windows


even with a 50% compression I think it will take 1hr * 48GB*2 / 7.77GB to finish
Approximately, 6 hours at 50% compression or up to 12 hours if the compression is already calculated.
and there are another 2 48 GB VM's after this.
Australia has 50GBit connection not like the 1000GBit in Slovakia

And before you put it down what if there is a corruption and you need to get back to a backup quickly?

Just a thought given this is a beta and I am sure you are planning.
Or even a multi thread upload - like Torrent - to flood the connection where you could select 1 thread or 10 threads.


It is good though - cant wait to see what daily / weekly / monthly etc... can do.


thanks
Damon

 

[damon1]

just noticed something when trying/looking to improve the performance or to assist others.

on the Proxmox nodes the back up is running, the backup now defaults to the selected but then changes to this


I think this is great as the BACKUP to PROXMOX are talking to each other.

Possibly get rid of this choice if the BACK UP is chosen or put a HELP to tell the user that the selection is redundant.

great job whoever did this


Side note : maybe a "performance connection test" so the USER will know what to expect from a initial upload of 10,40,80 GB drive

BUT - it is only for the initial upload as far as I can see (hope) WILL KNOW FOR SURE IN 12 HOURS


thanks
Damon

 

[damon1]

lastly,

if anyone is wondering about what memory or CPU are required



this is a slow i7 with 16GB of 1666MHz DDR3 memory

looks like the machine specs are not in the equation at all even with a ZFS drive.
The Network or WAN bandwidth is the bottleneck as expected.

thanks
damon

 

[dcsapak]

Hi,

could you upload via local media a copy of the initial VM.
Similar to CLONE but CLONE to USB

you can already do this, just copy (e.g. with rsync) the datastore to a disk (do not forget the '.chunks' folder) and then to the other server
but yes, we want to have a 'local sync' in the future (no timeframe yet though)

Side note : maybe a "performance connection test" so the USER will know what to expect from a initial upload of 10,40,80 GB drive

there is the command 'proxmox-backup-client benchmark' which tries to mimic a backup upload (the result for 'TLS speed' is the bandwidth over the network; the rest benchmarks the machine where the proxmox-backup-client is run)

 

[damon1]

HI Dominik,

thanks for the tip regarding rsync.
I am sure for someone who uses Linux even semi-regularly then this would be simple.
For me, and possibly other windows users, it is near impossible.
if I hadn't found MC (Midnight Command) then I would not be able to navigate the linux disk structure let alone edit a text file.


Yes, I found "proxmox-backup-client benchmark"

│ Name │ Value │
│ TLS (maximal backup upload speed) │ not tested │
│ SHA256 checksum computation speed │ 1725.10 MB/s (85%) │
│ ZStd level 1 compression speed │ 484.76 MB/s (64%) │
│ ZStd level 1 decompression speed │ 923.30 MB/s (77%) │
│ Chunk verification speed │ 609.01 MB/s (80%) │
│ AES256 GCM encryption speed │ 2808.71 MB/s (77%) │


thanks
damon