slow sync job PBS

[DEZERTIR]

We have an issue: the data transfer rate during sync job is extremely slow, 20-50 MB/s.

The hard drives used are a 3.5" Toshiba MG10-F 22 TB MG10SFA22TE SAS 12Gb/s hard drive.
Connected via a 12Gb/s HBA
There is no load on them during the data transfer process.

PBS version 3.4.1
Specifications:


sync job:





7 TB are transferred in 1 day 10 hours, this is very long for us

Transferring 7 TB took about 1 day and 10 hours, which is a very long time for us.
please help

 

[SteveITS]

What’s the network speed?

40 MBps is about 400 Mbps…what’s the real life transfer rate for that HDD?

Proxmox recommends SSD: https://pbs.proxmox.com/docs/instal...age:-,Prefer fast storage,-that delivers high

80% CPU seems high too but I did not look up that CPU model.

 

[DEZERTIR]

What’s the network speed?

40 MBps is about 400 Mbps…what’s the real life transfer rate for that HDD?

Proxmox recommends SSD: https://pbs.proxmox.com/docs/installation.html#:~:text=Backup storage:-,Prefer fast storage,-that delivers high

80% CPU seems high too but I did not look up that CPU model.

sync job run in local

12Gb/s

 

[DEZERTIR]

Please help

 

[IsThisThingOn]

What are your results from this: https://pve.proxmox.com/wiki/PBS_Client_CPU_Benchmark ?

 

[DEZERTIR]

What are your results from this: https://pve.proxmox.com/wiki/PBS_Client_CPU_Benchmark ?

 

[news]

Hi there, my results on an Proxmox all ZFS NAS with Intel LGA1700 System on
Intel Core-i3 12100T and BIOSTAR B760MX2-E PRO D4 is:

$ proxmox-backup-client benchmark
SHA256 speed: 1993.61 MB/s  
Compression speed: 593.39 MB/s  
Decompress speed: 966.87 MB/s  
AES256/GCM speed: 5508.90 MB/s  
Verify speed: 650.75 MB/s  
┌───────────────────────────────────┬─────────────────────┐
│ Name                              │ Value               │
╞═══════════════════════════════════╪═════════════════════╡
│ TLS (maximal backup upload speed) │ not tested          │
├───────────────────────────────────┼─────────────────────┤
│ SHA256 checksum computation speed │ 1993.61 MB/s (99%)  │
├───────────────────────────────────┼─────────────────────┤
│ ZStd level 1 compression speed    │ 593.39 MB/s (79%)   │
├───────────────────────────────────┼─────────────────────┤
│ ZStd level 1 decompression speed  │ 966.87 MB/s (81%)   │
├───────────────────────────────────┼─────────────────────┤
│ Chunk verification speed          │ 650.75 MB/s (86%)   │
├───────────────────────────────────┼─────────────────────┤
│ AES256 GCM encryption speed       │ 5508.90 MB/s (151%) │
└───────────────────────────────────┴─────────────────────┘

 

[Neobin]

Transferring 7 TB took about 1 day and 10 hours, which is a very long time for us.

Maybe do not use ancient (16 years: [1]) hardware and a single HDD then?

[1] https://www.intel.com/content/www/u...40-ghz-5-86-gts-intel-qpi/specifications.html

 

[IsThisThingOn]

View attachment 97702

run it again but this time with a PBS destination so you get a TLS result on PVE.
Then run the same benchmark again on PBS, where you don't get a TLS result.

 

[DEZERTIR]

the verification task slowed down the work, thanks

 

[SteveITS]

On an HDD, anything else running will noticeably slow down everything. So backup, sync, verify should not overlap I would think.