Redhat VirtIO developers would like to coordinate with Proxmox devs re: "[vioscsi] Reset to device ... system unresponsive"

[_gabriel]

Might try using Virtio SCSI

it's mandatory , as this is the point of the topic.

 

[band20]

it's mandatory , as this is the point of the topic.

didn't helps neither unfortunately.

 

[_gabriel]

then your problem is elsewhere.
that's not this topic.
slow destination backup can slowdown VM up to crash.
create new topic, or append to existing topics about it.

 

[band20]

Indeed, the problem has appeared soon after we changed the provider to Starlink which has very low upstream bandwidth and the backup we do is cloud based. so ineed that migh be a reason. thanks for the hint.

 

[_gabriel]

the recommended way for primary backup, is local backup, then secondary backup can sync from primary backup, which not disturb the VM.

 

[Interpipes]

Indeed, the problem has appeared soon after we changed the provider to Starlink which has very low upstream bandwidth and the backup we do is cloud based. so ineed that migh be a reason. thanks for the hint.

Assuming it is an option with however you are backing up, have you tried turning on backup fleecing? https://pve.proxmox.com/pve-docs/chapter-vzdump.html#_vm_backup_fleecing

 

[emunt6]

If you want permanent fix, already mentioned in this thread:
- Use SATA disk emulation with Default controller (LSI 53C895A),
- Do not use virtio- emulation in Windows VM,
- Other VMs works fine with virtio (Linux),

This can be fix up to Windows Server 2025,
for Windows Server 2025 there is no fix (There is no driver for legacy SATA controller - LSI 53C895A).
It will drop performance? Yes,but it will be stable and error free.

It is very sad to see, the upstream KVM/QEMU devs cannot solve this long exsisting problem, but other Virt. platform did ( Vmware )

I can answer the "Windows Server 2025" question: use "nvme disk" emulation subsystem:
PVE v8 , PVE v9 - will work ( requires qemu 9.2+ )

https://www.qemu.org/docs/master/system/devices/nvme.html

-drive file=disk0.qcow2,if=none,id=nvm
-device nvme,serial=deadbeef,drive=nvm

So in time, the problem solved (get rid of SAS/SATA emulation).

 

[carver.it]

Anyone here advise from that huge long thread what the current state of play is with MS SQL server and PVE?

My thread and details here: https://forum.proxmox.com/threads/sql-server-2022-database-corruption.185694/#post-865582

 

[emunt6]

Anyone here advise from that huge long thread what the current state of play is with MS SQL server and PVE?

My thread and details here: https://forum.proxmox.com/threads/sql-server-2022-database-corruption.185694/#post-865582

See #287
Use SATA disk emulation.

 

[SteveITS]

0.1.302 came out last week. We haven't tried it yet.
https://fedorapeople.org/groups/vir...ownloads/archive-virtio/virtio-win-0.1.302-1/
https://www.reddit.com/r/Proxmox/s/rIy1Ujf8D4

 

[proteus]

It fixes the SQL problem but not the TRIM one. The TRIM problem is only if you are using ZFS as I understand.
Version .271 is still the only one with none of these problems.

 

[_gabriel]

it's August Windows Update reveals trim error, not Virt SCSI driver.
Myself have trim failure since few years, with some configurations, only setting discard_granularity in args: fix it.

 

[proteus]

In .271 you do not need to set discard_granularity.

 

[_gabriel]

have you double checked ?

 

[SteveITS]

discard_granularity was still needed on driver .266, at least for the CPU usage/timeouts. We are using Ceph for VM storage.
ref:
2024 post: https://www.alldiscoveries.com/how-...frag-on-windows-server-virtualized-with-qemu/
and various threads here like: https://forum.proxmox.com/threads/server-2022-defrag-trim-pegging-cpu-and-ram.161514/post-818884
https://bugzilla.proxmox.com/show_bug.cgi?id=5552

 

[santiagobiali]

It fixes the SQL problem but not the TRIM one. The TRIM problem is only if you are using ZFS as I understand.
Version .271 is still the only one with none of these problems.

The TRIM issue occurs because Microsoft introduced some trim functions not yet implemented in virtio's driver. It's not related to ZFS's backend.

I've added what I found in this github issue: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/1574#issuecomment-5478445229

Only affects updated W11 and Windows Server 2022/2025.
Virtio 271, 285 or 302 doesn't make a difference in this TRIM issue, all fail.

 

[complexplaster27]

Hello I've have installed this new driver on our Test servers and have done the following in terms of multi-day testing:

• Boot-volume operation on vioscsi 0.1.302
• Multiple reboots and device checks
• Five active VirtIO SCSI controllers
• Overnight operational soak
• Multiple PBS snapshot backups
• Guest-agent filesystem freeze and thaw under heavy I/O
• 4 KiB random mixed read/write workloads
• 1 MiB sequential mixed read/write workloads
• Single-controller and dual-controller tests
• Up to 256 outstanding I/O requests
• More than 245 million DiskSpd operations
• Approximately 606 MiB/s sustained dual-controller sequential throughput
• A one-hour random workload completing 207.5 million operations
• Repeated successful DBCC CHECKDB checks
• No Event ID 129 RAID-port resets
• No Event ID 153 I/O retries
• No DiskSpd I/O errors
• No disk, Storport, vioscsi, NTFS, or Volmgr warnings
• No missing or unhealthy disks
• No SQL database consistency errors

I definitely couldn't force a raidport reset like I could previously, I will be doing more rigorous testing in the coming days trying my best to completely fry the test servers.