VM I/O errors on all disks

[kristian.kirilov]

use scsiX for your disks instead virtioX, and keep virtio-scsi controller type.

But this will change how the disk are recognized by the OS, which means - unable to boot the OS

 

[kristian.kirilov]

Actually I was wrong, because it uses UUID, the UUID remains the same, no matter you are using SCSI or VirtIO.
Will change it and check if the errors still appear.
Many thanks!

 

[shukko]

I'm not sure what you are talking about,

View attachment 31783

You want me to switch to Virtio Single or?

View attachment 31784

Hello

Your disk is now using virtio0

Select this disk and click detach from the top

then double click the unused disk which will be on bottom of the hardware list
and reattch it using scsi (default option)

 

[kristian.kirilov]

Yeah, already did it all good.
The issue disappeared!
Many thanks.

Bear in mind, Windows will go BSOD when doing such change. Or at least mine did it ;-)
Luckily I don't have any production Windows OSs.

 

[Kabbone]

Also works for me with Virtio SCSI, but like said before the Virtio Block also throws no error on my non NVMe LVM SSD

 

[kristian.kirilov]

Yeah, it seems that there is some incompatibility between ZFS and VirtIO Block devices

 

[tr_inett]

I can confirm the problem - after updating to Proxmox 7.1, there were also several Linux VMs in our cluster that were still running with the default SCSI controller (LSI 53C895A) and virtioX and which had "buffer I/O error" or "print_req_error: I/O error" in the log after the update; in one case, data corruption also occurred within the VM.

Changing the SCSI controller to "VirtIO SCSI" and the VM hard disks to "scsiX" also helped here.

 

[eds]

I had exactly the same problem, with an 2016 kvm machine being migrated to an new PVE 7 host.
The workaround with the SCSI controller seems to work.
Now I will have to find out, what/if data has been lost. I found out about this problem, because my clamav db was corrupted!

Maybe a moderator can pin this post and also add a chapter to the "known issues" when migrating to PVE7??
https://pve.proxmox.com/wiki/Upgrade_from_6.x_to_7.0#Known_upgrade_issues

 

[Funar]

This might help:
https://gitlab.com/qemu-project/qemu/-/issues/649

I ran into the same I/O errors on an iSCSI-backed Proxmox deployment, but my FibreChannel one seems unaffected. Downgrading back to QEMU 6.0.0 (pve-qemu-kvm 6.0.0-4) will resolve the issue without having to change from virtio to SCSI.

apt install pve-qemu-kvm=6.0.0-4

You'll have to shutdown any VMs you have running and then start them up again for them to be on 6.0.0 after doing this.

 

[kristian.kirilov]

Thanks @Funar so you can clearly say this is a bug.
Not sure how to make this official, I hope some of the Proxmox developers read the forums.

 

[Funar]

Thanks @Funar so you can clearly say this is a bug.
Not sure how to make this official, I hope some of the Proxmox developers read the forums.

It's definitely a bug. Not with Proxmox, but in QEMU itself. I've been running mine on the aforementioned pve-qemu-kvm-6.0.0-4 package without a single problem.

 

[tom]

Please re-test with current qemu package, see changelog:

http://download.proxmox.com/debian/...n/binary-amd64/pve-qemu-kvm_6.1.0-3.changelog

 

[kristian.kirilov]

I made the test this morning, and it seems to work fine for me. Made the same test as I did it in the past (trying to download gitlab-ce package which is almost 1GB of size). The error messages are not shown anymore.

Because my test is not comprehensive, please the other guys to test as well, and comment here.
Thanks ;-)

 

[nhand42]

Same issue here with a Debian container. I could reliably get I/O errors when building a development project inside the container. This caused the root partition to remount read-only. Was also using virtio block over ZFS storage. Upgraded the pve-qemu-kvm_6.1.0-3 package linked by Tom and the problems went away.