"After migrating about 20 VM's from Vware my ->>>> PBS ran backups <<<<<- @ about 200-300mb/s."
"I have a --->>>>>> Proxmox enviroment <<<<-- where i migrated my 20vms som VMware"
PBS ZFS disk speed thoughts option
- Thread starter AkUsP
- Start date
"After migrating about 20 VM's from Vware my ->>>> PBS ran backups <<<<<- @ about 200-300mb/s."
"I have a --->>>>>> Proxmox enviroment <<<<-- where i migrated my 20vms som VMware"
SHA256 speed: 479.43 MB/s
Compression speed: 488.70 MB/s
Decompress speed: 804.15 MB/s
AES256/GCM speed: 3460.90 MB/s
Verify speed: 298.84 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ not tested │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 479.43 MB/s (24%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed │ 488.70 MB/s (65%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed │ 804.15 MB/s (67%) │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed │ 298.84 MB/s (39%) │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed │ 3460.90 MB/s (95%) │
└───────────────────────────────────┴────────────────────┘
Did the test on my first machine also. About the same
Compression speed: 488.70 MB/s
Decompress speed: 804.15 MB/s
AES256/GCM speed: 3460.90 MB/s
Verify speed: 298.84 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ not tested │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 479.43 MB/s (24%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed │ 488.70 MB/s (65%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed │ 804.15 MB/s (67%) │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed │ 298.84 MB/s (39%) │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed │ 3460.90 MB/s (95%) │
└───────────────────────────────────┴────────────────────┘
Did the test on my first machine also. About the same
Run the benchmark on PVE with PBS as destination.
Then run the benchmark again on PBS.
Post both results
Then run the benchmark again on PBS.
Post both results
Last edited:
Then the TLS speed should be your bottleneck, since the PBS storage is too slow to write above 200MB/s. That is pretty bad for 2-4MB chunks IMHO.
But to be expected with a 6 wide RAIDZ2. You could switch to mirrors, that way you should be able to get 500MB/s, at which point other bottlenecks should arise like your CPU.
Ok will try tomorrow. What would the command look like to throw in on the PVE?
So you are suggesting making 1+1 mirror with 2 disks. Then throw all the 5 mirrors in a VDEV that in my world "looks" like a HW RAID10.
I noticed I can make the mirrors in the web gui. But is it correct that the VDEV parring still need to be done i ZFS tool in CLI ?
proxmox-backup-client benchmark --repository mybackupstore
For me, that was something like this:
Code:
proxmox-backup-client benchmark --repository 'test@pbs@[2001:XXXXXXXXXXXX::f691]:8007:poolBackup'Correct, I suggest you create multiple mirrors inside the same vdev. So basically a stripe over all the mirrors. Not sure how that works in the proxmox GUI to be honest. Works like this in CLI
Code:
zpool create backuppool mirror /dev/disk/by-id/ata-DISK1 /dev/disk/by-id/ata-DISK2 mirror /dev/disk/by-id/ata-DISK3 /dev/disk/by-id/ata-DISK42001:XXXXXXXXXXXX::f691 is that your IPV6 ?
After creating the VDEV pools using CLI. Will Proxmox GUI still show then with all info and stuff?
Yeah, my redacted IPv6
Jup, not a problem. Proxmox GUI just displays the pools, no matter how they are created. But you probably could also create it in the webGUI.
Last edited:
The PVE GUI will be fine for this, and that is how I created my six-drive RAID10 pool.
Assuming your PBS datastore is added to the PVE host, you should be able to just run this command I mentioned previously:
proxmox-backup-client benchmark --server yourpbsserverhere --repository yourdatastorenamehere
Assuming your PBS datastore is added to the PVE host, you should be able to just run this command I mentioned previously:
proxmox-backup-client benchmark --server yourpbsserverhere --repository yourdatastorenamehere
Cant get it right..
if we say Im in shell on one of me PVE:
My user on my pbs is jonnyknox@pbs
My IP on my PBS 192.168.1.100:8007
My datastore on PBS is called bigvolume
How should it look then?
if we say Im in shell on one of me PVE:
My user on my pbs is jonnyknox@pbs
My IP on my PBS 192.168.1.100:8007
My datastore on PBS is called bigvolume
How should it look then?
If you've added your PBS server to the PVE host via the GUI, then you can use my example.
If not, use IsThisThingOn's example with the username/etc. since the PVE host doesn't know where to go.
So:
If not, use IsThisThingOn's example with the username/etc. since the PVE host doesn't know where to go.
Code:
--repository <string>
Repository URL: [[auth-id@]server[:port]:]datastoreSo:
Code:
proxmox-backup-client benchmark --repository jonnyknox@pbs@192.168.1.100:8007:bigvolume
Last edited:
The PBS is added as trusted storage on the PVE cluster.
When i run the command with :8007 i says "error" cant find path.
When I remove the :8007, it asks for the password, as if the PBS user was not added on the PVE cluster.
I then enter the password for the PBS user. It then do something for 1-2 seconds. Then nothing.
Strange...
proxmox-backup-client benchmark --repository 'test@pbs@[2001:XXXXXXXXXXXX::f691]:8007oolBackup'
Not sure, but is it maybe just the missing ' ?
That could also be the case. I created the user on pbs, so that is why I am using that. I also just gave it full admin rights and deleted the user afterwards.