My VMs freeze and corrupt on high disk I/O

justarandomguy

New Member
Jul 21, 2019
9
0
1
26
Hi,
I am having a disk problem with Proxmox. I'm running two SATA 2TB hard drives connected in a RAID 1 ZFS. I'm running Proxmox 5.4-3 on an R710.
Sometimes, but not always, high disk I/O starts breaking things. For example, last night I was doing a disk import, which was causing my I/O delay to be around 15%. When the nightly backup started, the combination of I/O from both made it so the VMs couldn't use the disk. Some of these VMs were just debian servers that weren't even doing anything, yet I couldn't log into them because of disk bottleneck. On the disk I/O graph for the individual VMs, I can see usage drop to exactly 0.
After a few minutes of this, the VMs started to corrupt. Some complained about errors writing to the disk, and some said they hit a 120s timeout when accessing I/O. After a reboot, some VMs needed a manual fsck, but all needed an automatic one. This process repeated a few times before I decided to cancel the backup and the import(cancelling just the backup did not help)
Curious about what I can do about this for the future. I would prefer it if proxmox could handle high disk activity a little more gracefully!
 
I don't think that's the problem - I'm using an HBA and ECC memory.

Either your HBA is wrong for ZFS/buggy or your RAM is faulty. At least, that is the case almost all of such issues.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!