VMs remounting partition read-only and (Buffer) I/O errors since qemu 3.0

[jasgripen]

I think the problem lies in the combination detect_zeroes and discard.

I changed one of my vm:s config-file in /etc/pve... to “detect_zeroes=off,discard=on” and I haven't had any problem with that vm for several days.

Can anybody else confirm this? I don't have so high load on my cluster now so I don't know if it's just a coincidence.

 

[JRG]

Hello,

We are running a 3 node Proxmox cluster 6.4-6 (running kernel: 5.4.114-1-pve) (see details below) and a 3 node PetaSAN (2.7.3) cluster for CEPH and iSCSI (Proxmox and PetaSAN are connected using ceph, we are using iSCSI just for a hyper-v cluster)
A few days ago we noticed one of the VMs (ubuntu server 20.0.4) had a HDD failure which we couldn't recover (superblock was damaged) and all our attempts to fix this issue did not work we ended deleting the image and restoring it from backup. Unfortunately we couldn't get the logs from the VM cause were were unable to mount and/or explore the virtual HDD.
Maybe worth mention it but the versions of ceph on proxmox and petasan they didn't match (Proxmox was running Luminous and PetaSAN is running Nautilus)
We attempted several times to recreate the issue using FIO but we couldn't replicate it.
I was following this topic and I'm wondering if is this issue with qemu corrupting RBD images is an issue still on newest versions of Proxmox/QEMU cause it seems this issue is the same, I couldn't find any logs on Proxmox neither related with this issue.
Thank you in advance for your time.

proxmox-ve: 6.4-1 (running kernel: 5.4.114-1-pve)
pve-manager: 6.4-6 (running version: 6.4-6/be2fa32c)
pve-kernel-5.4: 6.4-2
pve-kernel-helper: 6.4-2
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.2-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.1.0
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.4-1
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.4-3
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.2-2
libpve-storage-perl: 6.4-1
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.1.6-2
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.5-5
pve-cluster: 6.4-1
pve-container: 3.3-5
pve-docs: 6.4-2
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-3
pve-ha-manager: 3.1-1
pve-i18n: 2.3-1
pve-qemu-kvm: 5.2.0-6
pve-xtermjs: 4.7.0-3
qemu-server: 6.4-2
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.4-pve1