Slow transfer/backup speeds to external USB Hard Disk (<2MB/s)

nothingness

New Member
Nov 22, 2020
1
0
1
Hi All,

I have build an energy friendly server with Proxmox VE (6.2) as virtualisation platform for our home setup (Home Automatisation, Nextcloud etc).

Although, I am not a Linux specialist I have managed setting up everything so-far by following a lot of tutorials (both written en video).

Now, it comes to (external) backups I have reached a point where I am completely stuck and I could use some assistance.

The proxmox server has an SSD on board + 4 * 5TB Seagate hard disks which are placed in a ZFS pool. Through the GUI it is possible to make scheduled backups of the VM’s to a directory on the ZFS pool. This works fine!

However, I also want to make external backups and this is where the problem occurs.

As media for the external backups, I have purchased 2 * 8TB Seagate USB Hard disks (to change every other week). These have been repartitioned/reformatted with gdisk and formatted .ext4.

To auto mount the USB drives I have installed “pve6-usb-automount”.

When testing the backups (GUI setup) to the external hard disks I noticed that this was extreme slow. At first, I thought it was related to backup system itself.

As a test I tried coping a backup (from a small VM) from the zfs pool to the usb hard disk with rsync –progress and this shows a speed blow 2 MB/s!

To make sure nothing is wrong with the USB hard disks and the rest of the hardware I have created a system rescue - USB stick (system-rescue.org ) to boot from. I download a number of ISO files to test speeds with “Grsync” copying the ISO’s between the drives and the speed was > 130MB/s.

It looks like it has something to do with the setup of Proxmox VE which limits the transfer speeds to the external usb drives, but I do not have clue what is.

According to “lsusb -t” the devices are connected with 5Gb/s

I have used google a lot to find solutions, but I was not able to find a solution, yet.

Your advice/tips are appreciated!

Kind regards,
Marco
 

wolfgang

Proxmox Staff Member
Staff member
Oct 1, 2014
6,483
454
103
Hi,

you can't compare rsync with a bulk copy.
If you are interested in what your USB Disk is capable use fio[1] instead.
When you backup a CT with rsync the first time you have to read/write many small (<4KB).
This is not the best pattern for speed :).

1.) https://fio.readthedocs.io/en/latest/fio_doc.html
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!