Hello.
I'm now experiencing freezes which started three days ago and happens every day but in random time. Hardware is Hetzner Server
Symptoms: Proxmox interface shows high IO delay, about 30-50%, while no significant IO operations are happening. I watched
Simple reboot fixes the problem until it happens again
In system logs i can see many records like this
Freezes come out of the blue: I didn't do any ZFS configurations recently, no new workloads too
So i thought it was faulty
But this time in logs all messages were about all the drives. Which makes me think, that that's not the drives case.
Some diagnostic info
Hardware Hetzner Server
CPU: Intel(R) Xeon(R) W-2295 CPU @ 3.00GHz
Memory: 4 * Samsung M393A4G43AB3-CWE ECC Registered + 4 * Samsung M393A4K40EB3-CWE ECC Registered Speed: 2933 MT/s
Drives:
Any help appreciated, thanks!
I'm now experiencing freezes which started three days ago and happens every day but in random time. Hardware is Hetzner Server
Symptoms: Proxmox interface shows high IO delay, about 30-50%, while no significant IO operations are happening. I watched
iotop, and 15M/s write was maximumSimple reboot fixes the problem until it happens again
In system logs i can see many records like this
Meanwhile
zpool events -v shows this
Code:
Apr 13 2026 15:54:06.617975577 ereport.fs.zfs.deadman
class = "ereport.fs.zfs.deadman"
ena = 0x819c1944f5406001
detector = (embedded nvlist)
version = 0x0
scheme = "zfs"
pool = 0xc0300e71306e37ec
vdev = 0xb685d326aa5b66c8
(end detector)
pool = "rpool"
pool_guid = 0xc0300e71306e37ec
pool_state = 0x0
pool_context = 0x0
pool_failmode = "wait"
vdev_guid = 0xb685d326aa5b66c8
vdev_type = "disk"
vdev_path = "/dev/sdd3"
vdev_ashift = 0x9
vdev_complete_ts = 0x4827c2f25336
vdev_delta_ts = 0xb40f810ebd
vdev_read_errors = 0x0
vdev_write_errors = 0x0
vdev_cksum_errors = 0x0
vdev_delays = 0x0
dio_verify_errors = 0x0
parent_guid = 0x79041166b7a9b905
parent_type = "mirror"
vdev_spare_paths =
vdev_spare_guids =
zio_err = 0x0
zio_flags = 0x300080 [CANFAIL DONT_QUEUE DONT_PROPAGATE]
zio_stage = 0x200000 [VDEV_IO_START]
zio_pipeline = 0x4e00000 [VDEV_IO_START VDEV_IO_DONE VDEV_IO_ASSESS DONE]
zio_delay = 0x0
zio_timestamp = 0x481318fd6e2e
zio_delta = 0x0
zio_type = 0x2 [WRITE]
zio_priority = 0x3 [ASYNC_WRITE]
zio_offset = 0x14922dd000
zio_size = 0x2000
zio_objset = 0x8245
zio_object = 0x1
zio_level = 0x0
zio_blkid = 0x1af7f7c
time = 0x69dcf57e 0x24d58f19
eid = 0x64Freezes come out of the blue: I didn't do any ZFS configurations recently, no new workloads too
So i thought it was faulty
/dev/sdd drive, i asked support to replace the drive, it helped, but not for long time - 5 hours later freezes repeated.But this time in logs all messages were about all the drives. Which makes me think, that that's not the drives case.
Some diagnostic info
Code:
zpool status -v
pool: rpool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: resilvered 4.47G in 00:00:20 with 0 errors on Mon Apr 13 22:55:05 2026
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdb3 ONLINE 0 0 0
sdc3 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
ata-Micron_5200_MTFDDAK1T9TDC_18501FD2E0D6-part3 ONLINE 0 0 0
sda3 ONLINE 0 0 0
errors: No known data errors
Code:
proxmox-ve: 9.1.0 (running kernel: 6.17.13-2-pve)
pve-manager: 9.1.7 (running version: 9.1.7/16b139a017452f16)
proxmox-kernel-helper: 9.0.4
pve-kernel-5.15: 7.4-15
proxmox-kernel-6.17: 6.17.13-2
proxmox-kernel-6.17.13-2-pve-signed: 6.17.13-2
proxmox-kernel-6.17.4-2-pve-signed: 6.17.4-2
proxmox-kernel-6.8: 6.8.12-17
proxmox-kernel-6.8.12-17-pve-signed: 6.8.12-17
pve-kernel-5.15.158-2-pve: 5.15.158-2
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph-fuse: 19.2.3-pve1
corosync: 3.1.10-pve2
criu: 4.1.1-1
frr-pythontools: 10.4.1-1+pve1
ifupdown2: 3.3.0-1+pmx12
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libproxmox-acme-perl: 1.7.1
libproxmox-backup-qemu0: 2.0.2
libproxmox-rs-perl: 0.4.1
libpve-access-control: 9.0.6
libpve-apiclient-perl: 3.4.2
libpve-cluster-api-perl: 9.1.1
libpve-cluster-perl: 9.1.1
libpve-common-perl: 9.1.9
libpve-guest-common-perl: 6.0.2
libpve-http-server-perl: 6.0.5
libpve-network-perl: 1.2.5
libpve-rs-perl: 0.11.4
libpve-storage-perl: 9.1.1
libspice-server1: 0.15.2-1+b1
lvm2: 2.03.31-2+pmx1
lxc-pve: 6.0.5-4
lxcfs: 6.0.4-pve1
novnc-pve: 1.6.0-3
proxmox-backup-client: 4.1.5-1
proxmox-backup-file-restore: 4.1.5-1
proxmox-backup-restore-image: 1.0.0
proxmox-firewall: 1.2.1
proxmox-kernel-helper: 9.0.4
proxmox-mail-forward: 1.0.2
proxmox-mini-journalreader: 1.6
proxmox-offline-mirror-helper: 0.7.3
proxmox-widget-toolkit: 5.1.9
pve-cluster: 9.1.1
pve-container: 6.1.2
pve-docs: 9.1.2
pve-edk2-firmware: 4.2025.05-2
pve-esxi-import-tools: 1.0.1
pve-firewall: 6.0.4
pve-firmware: 3.18-2
pve-ha-manager: 5.1.3
pve-i18n: 3.7.0
pve-qemu-kvm: 10.1.2-7
pve-xtermjs: 5.5.0-3
qemu-server: 9.1.6
smartmontools: 7.4-pve1
spiceterm: 3.4.1
swtpm: 0.8.0+pve3
vncterm: 1.9.1
zfsutils-linux: 2.4.1-pve1Hardware Hetzner Server
CPU: Intel(R) Xeon(R) W-2295 CPU @ 3.00GHz
Memory: 4 * Samsung M393A4G43AB3-CWE ECC Registered + 4 * Samsung M393A4K40EB3-CWE ECC Registered Speed: 2933 MT/s
Drives:
Code:
Device Model: SAMSUNG MZ7LH1T9HMLT-00005
Device Model: Micron_5200_MTFDDAK1T9TDC
Device Model: Micron_5200_MTFDDAK1T9TDC
Device Model: Micron_5200_MTFDDAK1T9TDCAny help appreciated, thanks!
Last edited:
