Backup issues with Windows VMs - 100% active time

vescovialex

Member
Aug 22, 2020
43
6
13
36
www.auxbit.it
I'm having some trouble with backups on my Proxmox cluster and I'm hoping someone can help me troubleshoot. I've been searching for solutions online for a while now, but I haven't been able to find anything that helps.

Here's my setup:

  • 3-node Proxmox cluster
  • Backups to both a NAS and a PBS
  • 10Gbps fiber networking (except for the NAS)
  • 10Gbps fiber-connected storage
  • ZFS over iSCSI storage
  • SSD and HDD pool on storage in raidz2

The problem:

Windows VM backups are taking significantly longer than usual. Backups that used to take 15 minutes now take 2.5 hours.


What I've tried:

  • Checked disk throughput: Disk throughput is high during backups, but it's also high during normal VM operation.
  • Checked I/O delay: I/O delay is normal (<1%).
  • Run scrubs on the ZFS pool: Scrubs have completed successfully.
  • Tested the SSDs: I've pulled the SSDs out of the storage and tested them separately, and they seem to be fine.

Additional observations:
  • Some Proxmox systems in my environment do not experience slow backups.
  • The disk on the backup target is pegged at 100% during backups.
  • The problem seems to have started recently (1-2 weeks ago).

I'm at a loss for what to try next. Any suggestions would be greatly appreciated.


Here are some additional details that may be helpful:

  • Proxmox VE version: 8.0.3
  • All disks on VMs are scsi
  • SCSI Controller: VirtIO SCSI
  • Virtio Guest: Installed
I'm hoping that someone in the community can help me figure out what's going on. Thanks in advance for your help!
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!