elastic vm showing bad block bitmap checksum on ext4

[dme31]

Hi !

I am requesting help !

My setup is :

Dell ME5 with 2 controllers ( mainly composed of 7200 rpm 16TB hard disk and 3.8TB ssd)

2 x 10GB switches aruba 8360-x48xtc v2 dédicated for the iscsi
3 x proxmox nodes 9.2.11 running on Hpe proliant dl360 gen11 server
Each with 2 x 10Gb nic 10Base-T BCM57416 with bnxt_en driver dédicated for the iscsi

Kernel Version 7.0.14-12-pve (2026-08-11T11:05Z)
multipath fully functionnal with all path running
Thick shared LVM built on top of the lun.
no iscsi timeout
no warnings on the san
firmware of the dell me5 up to date
firmwares of the dell me5 hard disk drive and ssd up to date
firmware of the HPe servers up to date.

The problem is that we have an Elastic cluster composed of 5 Hot nodes who are throwing the following error :

EXT4-fs error (device dm-7): ext4_validate_block_bitmap:423: comm elastic..][T#6]: bg 19130: bad block bitmap checksum
EXT4-fs (dm-7): Delayed block allocation failed for inode 76285244 at logical offset 8192 with max blocks 74 with error 74
EXT4-fs (dm-7): This should not happen!! Data will be lost

all five nodes are affected, and some others vms that are not running elastic but some others are totaly fine.
every time it is happening, we stop the elastic service, unmount the partition, run a fsck which clean the Filesystem and reboot the system.
the cluster goes green again until the same kind of error happens 2 or 3 days later and then we go again....

We have checked everything and disabled the offloading features of the bnxt driver without luck.

if you have any clue to help me sort out this issue i will be more than happy.

Thanks in advance for your advices

 

[bbgeek17]

Hi @dme31 , welcome to the forum.

Your situation might be related to this thread: PVE 9.2.x, and shared thick LVM on multipath, with guests reading back data different from what was written and nothing in the host or SAN logs. The summary of that thread is that the SAN lies about writes, leading to data corruption on the host.

There are a few things you can do to help with troubleshooting:
- post qm config for an affected VM and your storage.cfg entry (disk format, discard, aio, snapshot-as-volume-chain)
- the guest mount options and whether fstrim runs?

As a test, try disabling discard on those disks, and if possible reproduce on a scratch disk on the same storage with fio --verify=crc32c.

Keep in mind that fsck only repairs metadata and the “data will be lost” message means real data is affected, so make sure Elastic has replicas or snapshots elsewhere.

Of course if this is affecting your production the best step is to involve your vendors (Proxmox, Dell, etc).

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox