Slow backups to usb disk

[Jonathan8899]

I connected a usb drive to proxmox. I am using it to do nightly backups. The write speed is about 3MB/sec

IO delay seems to be peaking during backup hours.

How do i fix this?

This is the disk

https://www.westerndigital.com/en-k...y-passport-usb-3-0-hdd?sku=WDBPKJ0040BBK-WESN

I have this stat

root@proxmox:~# dd if=/dev/zero of=/mnt/pve/usbdisk/testfile bs=1M count=1000 oflag=direct
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 20.5477 s, 51.0 MB/s

And this is what I see for backups

INFO: backup is sparse: 9.34 GiB (29%) total zero data
INFO: transferred 32.00 GiB in 11347 seconds (2.9 MiB/s)
INFO: archive file size: 15.66GB

 

[cheiss]

Hi,

root@proxmox:~# dd if=/dev/zero of=/mnt/pve/usbdisk/testfile bs=1M count=1000 oflag=direct
1048576000 bytes (1.0 GB, 1000 MiB) copied, 20.5477 s, 51.0 MB/s

First of, writing zeros is not a benchmark of any kind, since most devices usually recognize streams of zero and just discards them. For example, the fio tool can be used for representative performance testing. But 51MB/s is slow even then.

Have you made sure the disk is connected via a USB 3.0 and uses that protocol? You can use e.g. dmesg -H and lsusb -v to get more details on the device as necessary.

The disk itself is spinning rust over USB, so perfomance probably won't be great in any case - PBS has to write a lot a very small files/chunks, and HDDs are just not made for such random-access usage.

 

[Jonathan8899]

Hi,


First of, writing zeros is not a benchmark of any kind, since most devices usually recognize streams of zero and just discards them. For example, the fio tool can be used for representative performance testing. But 51MB/s is slow even then.

Have you made sure the disk is connected via a USB 3.0 and uses that protocol? You can use e.g. dmesg -H and lsusb -v to get more details on the device as necessary.

The disk itself is spinning rust over USB, so perfomance probably won't be great in any case - PBS has to write a lot a very small files/chunks, and HDDs are just not made for such random-access usage.

Hello,

I just checked as you instructed. Yes it uses USB 3.0

 

[Jonathan8899]

Is a SSD usb disk better? or should I get something else?

 

[mr44er]

https://nascompares.com/answer/list-of-wd-cmr-and-smr-hard-drives-hdd/

WDBPKJ0040BBK = SMR drive

https://www.howtogeek.com/803276/cmr-vs.-smr-hard-drives-whats-the-difference/

SMR drives are the worst option for random-access...you could compare it with the sluggishness of an old USB(2.0)-stick.

Is a SSD usb disk better? or should I get something else?

If money is no problem, yes go for SSDs otherwise choose CMR HDDs. They perform also not "fast", but way better than your SMR disk.

 

[Jonathan8899]

When I look at the backup job details I see this

INFO: starting new backup job: vzdump --compress zstd --quiet 1 --all 1 --notes-template '{{guestname}}' --fleecing 0 --mode snapshot --prune-backups 'keep-last=7' --storage usbdisk

I read about changing the changing max-workers, would it make a difference?

Today I will also try uncompressed backup.

I also don't think this IO delay is normal

 

[mr44er]

would it make a difference?

No, SMR disks are slow.

I also don't think this IO delay is normal

For SMR disks it is.