Random IO Error - Windows Server 2025

[dcuadrados]

Hello everyone,

I'm experiencing a random "IO Error" that causes my two Windows Server 2025 Data Center VMs to randomly halt (yellow triangle in Proxmox). A reset/reboot resolves the issue temporarily.

My environment details are below. I suspect a potential conflict with my configuration, possibly related to I/O or the high RAM usage.


Node and Storage

Node: Proxmox VE 9.0.11 on Linux 6.14 kernel.
CPU: Intel Xeon E-2288G (16C).
RAM Usage: High (approx. 89% of 31 GiB).
Storage: ZFS pool built on two 960 GB Samsung NVMe SSDs (S.M.A.R.T. OK, low wearout).
Repo Status: Non production-ready repository enabled.


Windows VM Configuration

Both Windows Server 2025 VMs use the following critical settings:

Setting Value
SCSI Controller VirtIO SCSI single
Disk Image RAW format on ZFS
I/O Settings aio=io_uring, cache=writeback, discard=on, iothread=1, ssd=1
Memory 13 GiB and 6 GiB respectively
Processors Host CPU type
BIOS OVMF (UEFI)

Has anyone encountered this specific IO Error with this configuration (especially VirtIO/ZFS/IO_URING) on recent Proxmox versions?

My apologies, I accidentally posted this twice

 

[fiona]

Hi,
please check the host system logs/journal for any messages around the time of the issue. What does zpool status -v say?

What does the following output when the VM is in IO error state, replacing 123 with the actual ID:

echo '{"execute": "qmp_capabilities"}{"execute": "query-block"}' | socat - /run/qemu-server/123.qmp

?

 

[dcuadrados]

Hi,
please check the host system logs/journal for any messages around the time of the issue. What does zpool status -v say?

What does the following output when the VM is in IO error state, replacing 123 with the actual ID:

echo '{"execute": "qmp_capabilities"}{"execute": "query-block"}' | socat - /run/qemu-server/123.qmp

?

Right now there’s no IO Error; if it happens again, I’ll do it again. Anyway, I’ve already downgraded the driver to VirtIO version 0.1.271.

{"QMP": {"version": {"qemu": {"micro": 2, "minor": 1, "major": 10}, "package": "pve-qemu-kvm_10.1.2-1"}, "capabilities": []}}
{"return": {}}
{"return": [{"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "off", "active": true, "image": {"virtual-size": 3653632, "filename": "/usr/share/pve-edk2-firmware//OVMF_CODE_4M.secboot.fd", "format": "raw", "actual-size": 3653632, "dirty-flag": false}, "iops_wr": 0, "ro": true, "children": [{"node-name": "#block013", "child": "file"}], "node-name": "pflash0", "backing_file_depth": 0, "drv": "raw", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "/usr/share/pve-edk2-firmware//OVMF_CODE_4M.secboot.fd"}, "qdev": "/machine/system.flash0", "type": "unknown"}, {"device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "on", "active": true, "image": {"backing-image": {"virtual-size": 540672, "filename": "json:{\"driver\": \"raw\", \"size\": 540672, \"file\": {\"driver\": \"file\", \"filename\": \"/var/lib/vz/images/100102/vm-100102-disk-0.raw\"}}", "format": "raw", "actual-size": 664064, "dirty-flag": false}, "virtual-size": 540672, "filename": "json:{\"throttle-group\": \"throttle-drive-efidisk0\", \"driver\": \"throttle\", \"file\": {\"driver\": \"raw\", \"size\": 540672, \"file\": {\"driver\": \"file\", \"filename\": \"/var/lib/vz/images/100102/vm-100102-disk-0.raw\"}}}", "format": "throttle", "actual-size": 664064, "dirty-flag": false}, "iops_wr": 0, "ro": false, "children": [{"node-name": "f41fd0da37cb0538e56f7d2c231d098", "child": "file"}], "node-name": "drive-efidisk0", "backing_file_depth": 1, "drv": "throttle", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": true}, "file": "json:{\"throttle-group\": \"throttle-drive-efidisk0\", \"driver\": \"throttle\", \"file\": {\"driver\": \"raw\", \"size\": 540672, \"file\": {\"driver\": \"file\", \"filename\": \"/var/lib/vz/images/100102/vm-100102-disk-0.raw\"}}}"}, "qdev": "/machine/system.flash1", "type": "unknown"}, {"io-status": "ok", "device": "", "locked": false, "removable": true, "qdev": "ide2", "tray_open": true, "type": "unknown"}, {"io-status": "ok", "device": "", "locked": false, "removable": false, "inserted": {"iops_rd": 0, "detect_zeroes": "unmap", "active": true, "image": {"backing-image": {"virtual-size": 161061273600, "filename": "/var/lib/vz/images/100102/vm-100102-disk-1.raw", "format": "raw", "actual-size": 54190236160, "dirty-flag": false}, "virtual-size": 161061273600, "filename": "json:{\"throttle-group\": \"throttle-drive-scsi0\", \"driver\": \"throttle\", \"file\": {\"driver\": \"raw\", \"file\": {\"driver\": \"file\", \"filename\": \"/var/lib/vz/images/100102/vm-100102-disk-1.raw\"}}}", "format": "throttle", "actual-size": 54190236160, "dirty-flag": false}, "iops_wr": 0, "ro": false, "children": [{"node-name": "f289c781e188435fd6f4ab27d19d84b", "child": "file"}], "node-name": "drive-scsi0", "backing_file_depth": 1, "drv": "throttle", "iops": 0, "bps_wr": 0, "write_threshold": 0, "encrypted": false, "bps": 0, "bps_rd": 0, "cache": {"no-flush": false, "direct": false, "writeback": false}, "file": "json:{\"throttle-group\": \"throttle-drive-scsi0\", \"driver\": \"throttle\", \"file\": {\"driver\": \"raw\", \"file\": {\"driver\": \"file\", \"filename\": \"/var/lib/vz/images/100102/vm-100102-disk-1.raw\"}}}"}, "qdev": "scsi0", "type": "unknown"}]}