Hello,
I am looking for guidance on the safest next step for a single-node Proxmox VE host with a mirrored ZFS pool.
At first my priority was to preserve data and avoid destructive recovery. That is still preferred. However, at this point I am also open to destructive steps if necessary, as long as the goal is to stabilize the host and the risk/tradeoff is explicit.
The specific questions are:
1. Does this look like a known OpenZFS / Proxmox `async_destroy` / `bptree` / `spa_sync` / `txg_sync` hang pattern?
2. What would you do next from here?
3. If destructive action is the only realistic path, what is the least dangerous order of operations?
Environment
Storage topology
NVMe health
Both mirror members currently look healthy by SMART/NVMe:
Incident summary
Originally, `zpool status -v servertank` showed a permanent error on:
That zvol appears to belong to a VM that no longer exists:
I also confirmed:
- no process was using `/dev/zvol/servertank/vm-200-disk-1`
- no active `lsof` reference
- PBS still has a backup for VM 200:
What happened
First destroy attempt
I destroyed the orphaned zvol.
It initially looked successful because:
- `zpool status -v servertank` changed to `errors: No known data errors`
- `zfs list servertank/vm-200-disk-1` changed to `dataset does not exist`
But the `zfs destroy` caller never returned and remained stuck in `D` state.
Reboot before unwind
A reboot happened before that destroy fully unwound.
After reboot/import:
- the orphaned zvol reappeared
- the permanent error returned
So the first apparent destroy was not durable across reboot.
Second destroy attempt
I repeated the safety checks and destroyed the orphan again from a clean boot.
That produced the same pattern:
- pool currently shows `errors: No known data errors`
- dataset currently shows `dataset does not exist`
- but the destroy caller is still blocked and `txg_sync` is hung
Current pool state
Current stuck tasks
Blocked destroy stack:
ZFS debug evidence
The destroy appears to be accepted and logged:
Then repeated deadman messages:
Also one slow active I/O was reported:
TXG / tunable state
Tail of txg history:
These txgs appear to stop moving after the destroy event.
What I already tried
1. Confirmed the damaged object was an orphan, not a live VM disk.
2. Confirmed a PBS backup exists for VM 200.
3. Destroyed the orphan once.
4. Reboot happened before unwind; orphan came back.
5. Destroyed the orphan again from clean boot.
6. Kept backups disabled the whole time to avoid more churn.
7. Tried a runtime-only, reversible tuning change based on similar OpenZFS async-destroy issue reports:
I monitored for about 2 minutes, then restored it to `500`.
That did not help:
- `txg_sync` stayed blocked
- the destroy caller stayed blocked
- deadman warnings kept increasing
- pool state did not otherwise change
Backup posture
I intentionally kept Proxmox backups disabled while this is unresolved:
What I am asking for
Given this exact state, what would you do next?
I would especially value answers in one of these forms:
1. Preservation-first path — safest next steps if avoiding further risk is the priority.
2. Balanced path — reasonable next diagnostic/recovery actions with moderate risk.
3. Last-resort path — destructive steps you would take if this is unrecoverable by normal means.
If the answer is that some destructive action is now justified, that is acceptable — I just want the order of operations and risk reasoning to be explicit.
If there are exact commands you want me to run, I can provide the output.
Thanks.
[/CODE]
I am looking for guidance on the safest next step for a single-node Proxmox VE host with a mirrored ZFS pool.
At first my priority was to preserve data and avoid destructive recovery. That is still preferred. However, at this point I am also open to destructive steps if necessary, as long as the goal is to stabilize the host and the risk/tradeoff is explicit.
The specific questions are:
1. Does this look like a known OpenZFS / Proxmox `async_destroy` / `bptree` / `spa_sync` / `txg_sync` hang pattern?
2. What would you do next from here?
3. If destructive action is the only realistic path, what is the least dangerous order of operations?
Environment
Code:
Date: 2026-05-03T13:16:10-04:00
Hostname: server
OS: Debian GNU/Linux 13 (trixie)
Kernel: Linux 6.17.13-3-pve
Architecture: x86-64
Board: ASRock Z690M-ITX/ax
Firmware: 20.02 (2025-05-09)
proxmox-ve: 9.1.0 (running kernel: 6.17.13-3-pve)
pve-manager: 9.1.9 (running version: 9.1.9/ee7bad0a3d1546c9)
proxmox-kernel-helper: 9.0.4
proxmox-kernel-6.17: 6.17.13-3
libpve-common-perl: 9.1.11
CPU: 13th Gen Intel(R) Core(TM) i5-13500
CPUs visible: 20
RAM: 62 GiB
Swap: 8 GiBStorage topology
Code:
Boot/root: ext4 on LVM on 240G Kingston SATA SSD
Pool: servertank
Topology: mirror-0
Members:
- nvme-SPCC_M.2_PCIe_SSD_230086415180103_1
- nvme-SPCC_M.2_PCIe_SSD_4WUZ6F7JFS65QD6NVKT5_1
zpool list servertank:
NAME SIZE ALLOC FREE FRAG CAP HEALTH
servertank 952G 68.2G 884G 16% 7% ONLINENVMe health
Both mirror members currently look healthy by SMART/NVMe:
Code:
Drive 1:
Model Number: SPCC M.2 PCIe SSD
Serial Number: 230086415180103
SMART overall-health self-assessment test result: PASSED
Critical Warning: 0x00
Temperature: 42 Celsius
Available Spare: 100%
Percentage Used: 3%
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Drive 2:
Model Number: SPCC M.2 PCIe SSD
Serial Number: 4WUZ6F7JFS65QD6NVKT5
SMART overall-health self-assessment test result: PASSED
Critical Warning: 0x00
Temperature: 38 Celsius
Available Spare: 100%
Percentage Used: 13%
Media and Data Integrity Errors: 0
Error Information Log Entries: 0Incident summary
Originally, `zpool status -v servertank` showed a permanent error on:
Code:
servertank/vm-200-disk-1:<0x1>That zvol appears to belong to a VM that no longer exists:
Code:
qm config 200
Configuration file 'nodes/server/qemu-server/200.conf' does not existI also confirmed:
- no process was using `/dev/zvol/servertank/vm-200-disk-1`
- no active `lsof` reference
- PBS still has a backup for VM 200:
Code:
pbs:backup/vm/200/2026-01-30T07:04:24ZWhat happened
First destroy attempt
I destroyed the orphaned zvol.
It initially looked successful because:
- `zpool status -v servertank` changed to `errors: No known data errors`
- `zfs list servertank/vm-200-disk-1` changed to `dataset does not exist`
But the `zfs destroy` caller never returned and remained stuck in `D` state.
Reboot before unwind
A reboot happened before that destroy fully unwound.
After reboot/import:
- the orphaned zvol reappeared
- the permanent error returned
So the first apparent destroy was not durable across reboot.
Second destroy attempt
I repeated the safety checks and destroyed the orphan again from a clean boot.
That produced the same pattern:
- pool currently shows `errors: No known data errors`
- dataset currently shows `dataset does not exist`
- but the destroy caller is still blocked and `txg_sync` is hung
Current pool state
Code:
pool: servertank
state: ONLINE
scan: scrub repaired 0B in 00:13:44 with 3 errors on Sun Apr 12 00:37:45 2026
config:
NAME STATE READ WRITE CKSUM
servertank ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
nvme-SPCC_M.2_PCIe_SSD_230086415180103_1 ONLINE 0 0 0
nvme-SPCC_M.2_PCIe_SSD_4WUZ6F7JFS65QD6NVKT5_1 ONLINE 0 0 0
errors: No known data errors
Code:
zfs list -t volume servertank/vm-200-disk-1
cannot open 'servertank/vm-200-disk-1': dataset does not existCurrent stuck tasks
Code:
1044 D [txg_sync]
19122 Ss bash -c set -e; zfs destroy servertank/vm-200-disk-1 ...
19123 D zfs destroy servertank/vm-200-disk-1Blocked destroy stack:
Code:
[<0>] cv_wait_common+0xb0/0x140 [spl]
[<0>] __cv_wait_io+0x18/0x30 [spl]
[<0>] txg_wait_synced_flags+0xd9/0x160 [zfs]
[<0>] txg_wait_synced+0x10/0x60 [zfs]
[<0>] dsl_sync_task_common+0x22f/0x2f0 [zfs]
[<0>] dsl_sync_task+0x1a/0x30 [zfs]
[<0>] dsl_destroy_head+0x11c/0x1a0 [zfs]
[<0>] zfs_ioc_destroy+0xc7/0x1d0 [zfs]
[<0>] zfsdev_ioctl_common+0x8ab/0x970 [zfs]
[<0>] zfsdev_ioctl+0x57/0xf0 [zfs]ZFS debug evidence
The destroy appears to be accepted and logged:
Code:
set servertank/vm-200-disk-1 (id 1159) refreservation=0
destroy servertank/vm-200-disk-1 (id 1159) (bptree, mintxg=1)
bptree index 0: traversing from min_txg=1 bookmark 0/0/0/0Then repeated deadman messages:
Code:
slow spa_sync: started 2946 seconds ago, calls 39
slow spa_sync: started 3007 seconds ago, calls 40
slow spa_sync: started 3069 seconds ago, calls 41
slow spa_sync: started 3130 seconds ago, calls 42
slow spa_sync: started 3191 seconds ago, calls 43
slow spa_sync: started 3253 seconds ago, calls 44Also one slow active I/O was reported:
Code:
slow vdev: /dev/disk/by-id/nvme-SPCC_M.2_PCIe_SSD_230086415180103_1-part1 has 1 active IOsTXG / tunable state
Code:
zfs_free_min_time_ms=500
zfs_txg_history=100
zfs_deadman_failmode=wait
zfs_deadman_synctime_ms=600000
zfs_deadman_ziotime_ms=300000
zfs_deadman_checktime_ms=60000Tail of txg history:
Code:
1623992 ... C ...
1623993 ... S ...
1623994 ... O ...These txgs appear to stop moving after the destroy event.
What I already tried
1. Confirmed the damaged object was an orphan, not a live VM disk.
2. Confirmed a PBS backup exists for VM 200.
3. Destroyed the orphan once.
4. Reboot happened before unwind; orphan came back.
5. Destroyed the orphan again from clean boot.
6. Kept backups disabled the whole time to avoid more churn.
7. Tried a runtime-only, reversible tuning change based on similar OpenZFS async-destroy issue reports:
Code:
zfs_free_min_time_ms=0I monitored for about 2 minutes, then restored it to `500`.
That did not help:
- `txg_sync` stayed blocked
- the destroy caller stayed blocked
- deadman warnings kept increasing
- pool state did not otherwise change
Backup posture
I intentionally kept Proxmox backups disabled while this is unresolved:
Code:
[
{
"id" : "backup-daily-snapshot",
"enabled" : 0,
"mode" : "snapshot",
"schedule" : "21:30",
"storage" : "pbs"
}
]What I am asking for
Given this exact state, what would you do next?
I would especially value answers in one of these forms:
1. Preservation-first path — safest next steps if avoiding further risk is the priority.
2. Balanced path — reasonable next diagnostic/recovery actions with moderate risk.
3. Last-resort path — destructive steps you would take if this is unrecoverable by normal means.
If the answer is that some destructive action is now justified, that is acceptable — I just want the order of operations and risk reasoning to be explicit.
If there are exact commands you want me to run, I can provide the output.
Thanks.
[/CODE]
