Promox Loses ZFS after each reboot

[bwoodruff]

PBS 3.0-2
SAS HBA in IT mode
Netapp DS2242 as a JBOD disk shelf

I can setup a Raidz3 pool, and everything works fine. Backups are made, etc. But the moment I reboot or shutdown and restart the server, the ZFS pool get messed up somehow and shows it's suspended. If I try running `zpool clear`, I lose access to the terminal and the web interface for several minutes, with no change. I've reformatted and rebuilt the ZFS pool from scratch 4 times now, and after the next reboot, it's back to this... No drive show any faults, and zpool reports all the drives are online. I've also eplaced the HBA with no change in behavior.

root@pbs:~# zpool status
pool: pbs-DataStore
state: SUSPENDED
status: One or more devices are faulted in response to IO failures.
action: Make sure the affected devices are connected, then run 'zpool clear'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-HC
config:

NAME STATE READ WRITE CKSUM
pbs-DataStore ONLINE 0 0 0
raidz3-0 ONLINE 2 24 0
sdb ONLINE 3 4 0
sdc ONLINE 3 4 0
sdd ONLINE 3 2 0
sde ONLINE 3 2 0
sdf ONLINE 3 4 0
sdg ONLINE 3 4 0
sdh ONLINE 3 2 0
sdi ONLINE 3 2 0
sdj ONLINE 3 4 0
sdk ONLINE 3 4 0
sdl ONLINE 3 4 0
sdm ONLINE 3 4 0
sdn ONLINE 3 8 0
sdo ONLINE 3 8 0
sdp ONLINE 3 6 0
sdq ONLINE 3 6 0
sdr ONLINE 3 2 0
sds ONLINE 3 2 0
sdt ONLINE 3 2 0
sdu ONLINE 3 2 0
sdv ONLINE 3 8 0
sdw ONLINE 3 8 0
sdx ONLINE 3 8 0
sdy ONLINE 3 8 0
errors: List of errors unavailable: pool I/O is currently suspended

errors: 4 data errors, use '-v' for a list

 

[bwoodruff]

Not using the GUI to create the ZFS pool, and using:
`zpool create <poolname> raidz3 <list of drives to use from /dev/disk/by-id/[uuid]>`

then running:
`zfs set mountpoint=/mnt/<mountpoint> <poolname>`

Improves the situation, upon rebooting the pool is suspended, but `zpool clear <poolName>` clears the issue and brings the pool online and usable., but this has to be done every single time the server is booted.

Anyone else run into this, and found a fix?