Correcting Storage

N

nleistad

Renowned Member
Jan 29, 2013
6
0
66
I have a Windows Server VM that experiences data corruption in the MSSQL tables of our CMS software. Software vendor blames the disk, SAN, RAID, Firmware.

Our Proxmox (v 9.1.7) is backed by an IXSystems M30 TrueNAS Scale SAN (10G connected storage network) served by NFS.

I now believe that I mistakenly created qcow2 disks for the VM clients and wonder if part of my issues stem from qcow2 stacked on top of ZFS' COW?

It looks like I can migrate disks and convert from qcow2 to raw. Would like a little confirmation that this is the correct path to take.

Proxmox Host:
CPU:Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz
Cores:40 (80 CPUs)
Memory: 512G

Hosting 9 other VMs, 3 Windows Server 2022, rest Linux, relatively light duty.
 
Hi @nleistad,

Confidently attributing the issue to NFS, QCOW, ZFS, network, or any other component requires proper analysis. This typically includes log review, reproducible testing, and potentially network trace reading.

There are known edge cases with QCOW on NFS, particularly around snapshot operations. Additionally, MS SQL is highly sensitive to I/O latency and consistency, so it’s possible the underlying infrastructure is simply not meeting its performance requirements.

That said, there is an ongoing discussion you should be aware of:
Y

Thread 'Redhat VirtIO developers would like to coordinate with Proxmox devs re: "[vioscsi] Reset to device ... system unresponsive"'

vrozenfe_at_redhat_dot_com (Vadim Rozenfeld) with Redhat would like to connect with Proxmox developers regarding a particular point of pain that Proxmox users seem to be having with their IO drivers: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/623#issuecomment-1875076293

This thread highlights issues around VirtIO SCSI drivers where “reset to device” events can occur, sometimes leading to unresponsive systems or degraded disk behavior, especially under certain I/O patterns or workloads like backups or SQL.

I suspect you are using VirtIO drivers, if so, it would be worth verifying the exact version in use, as some versions have known regressions and downgrades have been used as a mitigation in certain cases.

As to whether moving to RAW type on NFS will be helpful - not if its the VirtIO issue. You will just have to give it a try. Note that the thread I referenced contains several ways to reliably reproduce the issue, so you may want to try that with different storage types.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
You must log in or register to reply here.
Top Bottom
Back