Low backup speed: Proxmox Backup to OMV NAS via NFS

[DonSiffo]

Hey everyone, I am new to the whole home server topic but I am somehow familiar with the basics I guess.

So I am running the following setup:

Latest Proxmox and OMV version on both systems.

OMV Backup NAS Server:
Raspberry Pi 4B + external Raid0 enclosure (2x 2 TB WD Red, no SMR drivex, = 4 TB with Raid0 Hardware Raid with Raidcontroller from external raid enclosure)

Filesystem is ext4. Connected via USB 3. Dedicated to be the Backup Server for the Proxmox VE.



Proxmox Server:
Running with a Intel Nuch with 8 GB Ram. Boot is a 250 GB internal SSD. Data Storage is a external 4TB HDD (ZFS) connected over USB 3.


Both run with gigabit lan over the router.


NFS is connected and mounted within Proxmox storage. Backup is also done via Snapshot and vzdump --compress lzo . BUT it took 3,5 h for ~40 GB and 2 VMs what looks in my eyes way too long. I also can not really interprate the logs. So I wanted to share them with asking for some guidance. Dont know what 0 write means for example. And maybe you have some advices how to improve speed or why it is so slow maybe due to a bad configuration or setting.

CPU load is no issue on both system. Netwoork seems ok. OMV Backup System has long time zero load then cpu jumps to 40-50 I hear the drive writing for a second and then again long time pause what seems also like a pattern.

Here is the log:

Spoiler: Log

INFO: starting new backup job: vzdump --compress lzo --all 1 --mailnotification failure --quiet 1 --storage Backup --mode snapshot --mailto XXX
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2020-05-03 03:00:02
INFO: status = running
INFO: update VM 100: -lock backup
INFO: VM Name: OMV
INFO: include disk 'virtio0' 'local-lvm:vm-100-disk-0' 12G
INFO: include disk 'virtio1' 'WDext:vm-100-disk-0' 3000G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/pve/Backup/dump/vzdump-qemu-XXX'
INFO: started backup task 'XXX'
INFO: status: 0% (349700096/3234110373888), sparse 0% (126222336), duration 3, read/write 116/74 MB/s
INFO: status: 1% (32400015360/3234110373888), sparse 0% (12493316096), duration 467, read/write 69/42 MB/s
INFO: status: 2% (64906133504/3234110373888), sparse 1% (37471936512), duration 733, read/write 122/28 MB/s
INFO: status: 3% (97100103680/3234110373888), sparse 2% (69664305152), duration 838, read/write 306/0 MB/s
INFO: status: 4% (129631780864/3234110373888), sparse 3% (102194380800), duration 939, read/write 322/0 MB/s
INFO: status: 5% (162018361344/3234110373888), sparse 4% (134580899840), duration 1041, read/write 317/0 MB/s
INFO: status: 6% (194239004672/3234110373888), sparse 5% (166801481728), duration 1146, read/write 306/0 MB/s
INFO: status: 7% (226472755200/3234110373888), sparse 6% (199035170816), duration 1247, read/write 319/0 MB/s
INFO: status: 8% (258777284608/3234110373888), sparse 7% (231339638784), duration 1349, read/write 316/0 MB/s
INFO: status: 9% (291343695872/3234110373888), sparse 8% (263905988608), duration 1452, read/write 316/0 MB/s
INFO: status: 10% (323584196608/3234110373888), sparse 9% (296144887808), duration 1552, read/write 322/0 MB/s

[....] ... 3 h later

INFO: status: 96% (3104905887744/3234110373888), sparse 94% (3053472854016), duration 10915, read/write 316/0 MB/s
INFO: status: 97% (3137423015936/3234110373888), sparse 95% (3085989920768), duration 11018, read/write 315/0 MB/s
INFO: status: 98% (3169608663040/3234110373888), sparse 96% (3118175506432), duration 11120, read/write 315/0 MB/s
INFO: status: 99% (3202025717760/3234110373888), sparse 97% (3150592495616), duration 11224, read/write 311/0 MB/s
INFO: status: 100% (3234110373888/3234110373888), sparse 98% (3182677082112), duration 11324, read/write 320/0 MB/s
INFO: transferred 3234110 MB in 11324 seconds (285 MB/s) INFO: archive file size: 40.81GB
INFO: Finished Backup of VM 100 (03:08:48) INFO: Backup finished at 2020-05-03 06:08:50
INFO: Starting Backup of VM 101 (lxc) INFO: Backup started at 2020-05-03 06:08:50
INFO: status = running
INFO: CT Name: Pi-Hole
INFO: backup mode: snapshot INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump' Logical volume "snap_vm-101-disk-0_vzdump" created.
INFO: creating archive '/mnt/pve/Backup/dump/vzdump-lxc-XXX' INFO: Total bytes written: 1051975680 (1004MiB, 56MiB/s)
INFO: archive file size: 496MB
INFO: remove vzdump snapshot Logical volume "snap_vm-101-disk-0_vzdump" successfully removed
INFO: Finished Backup of VM 101 (00:00:22)
INFO: Backup finished at 2020-05-03 06:09:12
INFO: Backup job finished successfully
TASK OK

 

[Alwin]

INFO: creating archive '/mnt/pve/Backup/dump/vzdump-lxc-XXX' INFO: Total bytes written: 1051975680 (1004MiB, 56MiB/s)

~56 MiB/s is a very good result. WD Red get 25 MB/s sequential write at best. And having it connected through a USB 3.0 RAID controller (simple).
In short, it is as good as it can get.

 

[DonSiffo]

~56 MiB/s is a very good result. WD Red get 25 MB/s sequential write at best. And having it connected through a USB 3.0 RAID controller (simple).
In short, it is as good as it can get.

One last thing: does it make any difference what filesystem the backup server (NAS) uses where the proxmox server is writing the backup to. It is a ext4 but I also could make it to zfs. Would this make any difference regarding the backup speed or something. Sorry for this (maybe) stupid question. I am not an expert regarding filesystems.

 

[Alwin]

You will need to try. But ZFS and RAID don't go well togehter.