On boot failed to start Import ZFS pool that was removed

[cbapel]

I get the following message on boot:

[FAILED] Failed to start Import ZFS pool pve\x2dvm01

IIRC, first, I removed the storage from the "data center", then I deleted this ZFS pool pve\x2dvm01 (or pve-vm01) using the destroy command instead of the GUI disk > ZFS > more. I then wiped the disks from the GUI disk menu to remove As a result, I suppose some operation did not take place and the system is trying to import so ghost pool.

root@ch01-pve01:~# systemctl --failed
  UNIT                           LOAD   ACTIVE SUB    DESCRIPTION
● zfs-import@pve\x2dvm01.service loaded failed failed Import ZFS pool pve\x2dvm01

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
1 loaded units listed.

Looking at the import cache seems okay

root@ch01-pve01:~# zpool get cachefile
NAME   PROPERTY   VALUE      SOURCE
rpool  cachefile  -          default
zfs01  cachefile  none       local


root@ch01-pve01:~# systemctl status zfs-import-cache
● zfs-import-cache.service - Import ZFS pools by cache file
     Loaded: loaded (/lib/systemd/system/zfs-import-cache.service; enabled; vendor preset: enabled)
     Active: active (exited) since Mon 2023-01-30 11:20:48 CET; 1h 15min ago
       Docs: man:zpool(8)
    Process: 2270 ExecStart=/sbin/zpool import -c /etc/zfs/zpool.cache -aN $ZPOOL_IMPORT_OPTS (code=exited, status=0/SUCCESS)
   Main PID: 2270 (code=exited, status=0/SUCCESS)
        CPU: 5ms

Jan 30 11:20:48 ch01-pve01 systemd[1]: Starting Import ZFS pools by cache file...
Jan 30 11:20:48 ch01-pve01 zpool[2270]: no pools available to import
Jan 30 11:20:48 ch01-pve01 systemd[1]: Finished Import ZFS pools by cache file.

The zpools appear okay

root@ch01-pve01:~# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
rpool  1.73T  1.71G  1.73T        -         -     0%     0%  1.00x    ONLINE  -
zfs01  3.48T  1.50M  3.48T        -         -     0%     0%  1.00x    ONLINE  -

root@ch01-pve01:~# zpool status -v
  pool: rpool
 state: ONLINE
config:

        NAME                                                 STATE     READ WRITE CKSUM
        rpool                                                ONLINE       0     0     0
          mirror-0                                           ONLINE       0     0     0
            nvme-eui.36363630549014930025385300000001-part3  ONLINE       0     0     0
            nvme-eui.36363630549014840025385300000001-part3  ONLINE       0     0     0

errors: No known data errors

  pool: zfs01
 state: ONLINE
config:

        NAME                                                STATE     READ WRITE CKSUM
        zfs01                                               ONLINE       0     0     0
          mirror-0                                          ONLINE       0     0     0
            nvme-SAMSUNG_MZQL23T8HCLS-00A07_S64HNC0TA04423  ONLINE       0     0     0
            nvme-SAMSUNG_MZQL23T8HCLS-00A07_S64HNC0TA04426  ONLINE       0     0     0

errors: No known data errors

However, this looks suspicious because it shows the ghost pool

root@ch01-pve01:~# ls /etc/systemd/system/zfs-import.target.wants/
 zfs-import-cache.service  'zfs-import@pve\x2dvm01.service'   zfs-import-scan.service   zfs-import@zfs01.service

When I try to remove this, I get an error that the file doesn't exist (maybe the path is wrong because I'm not doing the hexadecimal right).

root@ch01-pve01:~# rm /etc/systemd/system/zfs-import.target.wants/pve\x2dvm01.service
rm: cannot remove '/etc/systemd/system/zfs-import.target.wants/pvex2dvm01.service': No such file or directory

I've also done the following to clear the cache:

root@ch01-pve01:~# systemctl stop zfs-import-cache
root@ch01-pve01:~# rm -rf /etc/zfs/zpool.cache
root@ch01-pve01:~# systemctl start zfs-import-cache
root@ch01-pve01:~# systemctl is-enabled zfs-import-cache

Any ideas on how I can resolve this?

 

[fabian]

the file is called 'zfs-import@pve\x2dvm01.service', not 'pve\x2dvm01.service'. a simple systemctl disable zfs-import@pve\x2dvm01.service should also do the trick.

 

[cbapel]

Thanks for the support. Below is what I did; I still have the same error on boot. I'm sure I've missed something simple, so my pre-emptive apologies. I also find it weird that the "-" in zfs-import stays dashy and but the pool is hexadecimal \x2d

I can view all the files in the directory with nano, except the problem file. Should I copy the files, scrap the directory, and re-create?

root@ch01-pve01:~# systemctl disable zfs-import@pve\x2dvm01.service

root@ch01-pve01:~# ls /etc/systemd/system/zfs-import.target.wants/
 zfs-import-cache.service  'zfs-import@pve\x2dvm01.service'   zfs-import-scan.service   zfs-import@zfs01.service

root@ch01-pve01:~# rm /etc/systemd/system/zfs-import.target.wants/zfs-import@pve\x2dvm01.service
rm: cannot remove '/etc/systemd/system/zfs-import.target.wants/zfs-import@pvex2dvm01.service': No such file or directory

root@ch01-pve01:~# rm /etc/systemd/system/zfs-import.target.wants/zfs-import@pve-vm01.service
rm: cannot remove '/etc/systemd/system/zfs-import.target.wants/zfs-import@pve-vm01.service': No such file or directory

root@ch01-pve01:~# systemctl --failed
  UNIT                           LOAD   ACTIVE SUB    DESCRIPTION
● zfs-import@pve\x2dvm01.service loaded failed failed Import ZFS pool pve\x2dvm01

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
1 loaded units listed.

 

[fabian]

the "rm" command needs escaping or quoting, like so

rm '/etc/systemd/system/zfs-import.target.wants/zfs-import@pve\x2dvm01.service'

 

[cbapel]

It worked, and I've learned. Thank you, Fabian!

root@ch01-pve01:~# rm '/etc/systemd/system/zfs-import.target.wants/zfs-import@pve\x2dvm01.service'
root@ch01-pve01:~# ls /etc/systemd/system/zfs-import.target.wants/
zfs-import-cache.service  zfs-import-scan.service  zfs-import@zfs01.service