how do I solve a datastore missing each reboot?

B

bunk3m

New Member
Feb 22, 2023
3
1
1
In front of my computer in Canada
Hi everyone. (newbie alert)

I have a PBS 3.3.2 server on bare metal. There are two datastores used for backup of a PVE server.

I had a couple of drives and decided to stripe two 8TB drives when I started.

I wasn't able to create the stripe using the gui, so I did it via the command line with
Code: 
zpool create -f -o ashift=12 backup sdc sdd

This worked fine until I did an update of PBS in around November and the first backup failed as it couldn't "open chunk store".

After reading the forums, I found that the disks were OK just not showing up in the "zpool list" or "zpool status".

I followed directions from other posts and found the disks using
Code: 
ls -lah /dev/disk/by-id/
and added it back using
Code: 
zpool import -a -d /dev/disk/by-id/actual_disk_ID

The datastore was available and has worked for a few months.

I thought I fixed the problem but when I ran the next update yesterday and rebooted, I have exactly the same problem. I used the same steps to add back the disk. The second datastore on the pbs was created using the GUI and doesn't have this problem. I suspect that in doing the manual datastore creation I missed something or made a mistake but I don't know where to start to figure it out.

Could you provide some suggestions on what I have to look for to fix this properly so each reboot I don't have to manually add back the datastore?

Thanks in advance.
 
Last edited:
Just do "zpool import -d /dev/disk/by-id/" or when error with poolname backup too.
Then reboot the server to check as should be ok then.
 
waltar said:
Just do "zpool import -d /dev/disk/by-id/" or when error with poolname backup too.
Then reboot the server to check as should be ok then.
Click to expand...
Thank you @waltar

I tried to do the "zpool import -d /dev/disk/by-id/" adding the diskID at the end. Then I rebooted.
When the pbs server restarted, the pool wasn't available with same error about chunk storage.

I tried with "zpool import backup" only which mounted the pool again.
When I reboot, the pool is again missing with same error.

The error continues to be:
Bash: 
Bad Request (400)
unable to open chunk storage at "/backup/.chunks" - No such file or directory (os error 2)

I'm assuming I didn't understand what you meant as I have to manually import the pool called "backup" at each reboot. As I mentioned earlier, the only difference between this and the other pool was that this was created with the command line and is a multi-disk stripe where the other pool was created by the gui and has only one disk in the stripe.
 
Hi,
bunk3m said:
I tried with "zpool import backup" only which mounted the pool again.
When I reboot, the pool is again missing with same error.
Click to expand...
you have to make sure the pool is imported automatically at startup. This can be achieved by either setting the cachefile for the zpool e.g. by following and adapting the commands here to your need https://forum.proxmox.com/threads/z...-with-cache-file-how-to-fix.61925/post-284032
or use a systemd unit to import the pool by e.g. systemctl enable zfs-import@backup.service.
 
waltar said:
Look at your zfs services for system boot:
systemctl -a|grep zfs
You should even have exactly named this kind of link:
l /etc/systemd/system/zfs-import.target.wants/zfs-import@backup.service
lrwxrwxrwx 1 root root 39 Apr 15 2024 /etc/systemd/system/zfs-import.target.wants/zfs-import@backup.service -> /lib/systemd/system/zfs-import@.service
Click to expand...
Thank you @Chris and @waltar .

When I run "systemctl -a|grep zfs" I see
Bash: 
# systemctl -a|grep zfs
...
  zfs-import-cache.service          loaded    active   exited    Import ZFS pools by cache file
  zfs-import-scan.service           loaded    inactive dead      Import ZFS pools by device scanning
  zfs-import@backup2.service        loaded    active   exited    Import ZFS pool backup2

So both the "zfs-import-scan.service is dead and there is no "zfs-import@backup.service".

And

Bash: 
# systemctl status zfs-import-scan
○ zfs-import-scan.service - Import ZFS pools by device scanning
     Loaded: loaded (/lib/systemd/system/zfs-import-scan.service; enabled; preset: disabled)
     Active: inactive (dead)
  Condition: start condition failed at Mon 2025-02-03 19:28:21 EST; 23h ago
             └─ ConditionFileNotEmpty=!/etc/zfs/zpool.cache was not met
       Docs: man:zpool(8)

Feb 03 19:28:21 pbs systemd[1]: zfs-import-scan.service - Import ZFS pools by device scanning was skipped because of an unmet condition >
lines 1-8/8 (END)

I created the zpool.cache & did the "update-initramfs -k all -u" but the zpool.cache wasn't there after the reboot.

I recreated the zpool.cache file again and I see it under "/etc/zfs/". It has the right time and date stamp.

I ran "update-initramfs -k all -u" again.

After the reboot, the zpool.cache file is there but the zpool "backup" didn't mount.

So finally, I tried "systemctl enable zfs-import@backup.service" and this finally worked and allowed both zpools to mount on a reboot.

Thanks for your help.
 
  • Like
Reactions: waltar
You must log in or register to reply here.
Top Bottom