After update to PBS 0.8.9 one disk of ZFS RaidZ2 is missing

[alfe]

Hello,

I installed PBS Beta from the ISO (0.8.6) and updated to 0.8.9, but now one disk of my ZFS RaidZ2 is missing while not being actually removed (it worked before running apt update). I tried to re-add it but it says that the disk is already an active member of my pool:

~# zpool status
  pool: rpool
 state: ONLINE
status: One or more devices could not be used because the label is missing or
    invalid.  Sufficient replicas exist for the pool to continue
    functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: none requested
config:

    NAME                                            STATE     READ WRITE CKSUM
    rpool                                           ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        ata-TOSHIBA_MG06ACA800E_4030A04UFKRE-part3  ONLINE       0     0     0
        ata-TOSHIBA_MG06ACA800E_4030A05LFKRE-part3  ONLINE       0     0     0
        ata-TOSHIBA_MG06ACA800E_1060A0X5FKRE-part3  ONLINE       0     0     0
        8852051436562280351                         UNAVAIL      0     0     0  was /dev/sdd

I tried to use wipefs to remove all partitions and repartition it using sgdisk and then replace the disk using zfs replace but it still claims that the disk is an active member of the array. It seems like this is related to this bug: https://github.com/openzfs/zfs/issues/2076

 

[fiona]

Hi,
did you already try the workarounds mentioned in the bug report? As zpool labelclear is broken in this case and wipefs also doesn't seem to be good enough, I'd recommend trying the workaround mentioned at the end.

 

[alfe]

Yes, I tried that, but it didn't help after getting a "device is already part of the pool" and "device is busy". I reinstalled PBS with a ZFS Raid10 but after a reboot, I got a Grub rescue shell. After finding out that Proxmox has problems with HP servers (this is a Microserver Gen8), I installed an LSI HBA and now all the problems are gone. It seems like the integrated controller/firmware of HP servers messes with UUIDs and device names: https://forum.proxmox.com/threads/grub-errors-hp-servers.62009/

What is strage is that 0.8.6 from the ISO installed fine, but after doing a apt dist-upgrade to update all packages and doing a reboot to 0.8.9 messed up my system.