PBS Host Migration: Orphan Datastore

[snowturtle]

Hi All,
The hardware on my PBS system recently died (PSU death on OEM system) and so I've decided to go with new hardware on two systems instead of one.
My current system uses a total of 8 drives in 2 ZFS datastores, 2 M.2's and 6 HDDs; however, the first of the new systems only supports the 2 M.2's + boot drive.

My plan was just put the old OS drive and the 2 M.2's into the first new host and then build out another host for the 6 HDDs.

What I'm unsure about is how best to go about removing the datastore that the HDDs are tied to from this instance of PBS.
I'm thinking moving the HDDs to the new host with a fresh PBS install will be fine, I just do the PBS setup and then import the ZFS pool. Do let me know if there's a problem with that.
Now I'm not sure if there's anything 'special' I'd need to do on the existing PBS instance to disconnect the HDD datastore from the system. Can I just disconnect/remove it without the drives connected (and therefore with the datastore offline) or will that cause errors/issues which require me to first connect the datastore in order to disconnect/remove it?

I do have spare hardware as a test bench that I can use in order to boot the PBS instance with all 8 drives connected, if that's something I need to do.

Thanks for any replies I receive

 

[Dunuin]

You didn't used the SSDs as special devices or parts of these as metadata-only-L2ARC to boost the performance of the HDDs? This would have increased the GC performance by magnitudes and a little bit with backup/restore/verify.

 

[wahmed]

Hi All,
The hardware on my PBS system recently died (PSU death on OEM system) and so I've decided to go with new hardware on two systems instead of one.
My current system uses a total of 8 drives in 2 ZFS datastores, 2 M.2's and 6 HDDs; however, the first of the new systems only supports the 2 M.2's + boot drive.

My plan was just put the old OS drive and the 2 M.2's into the first new host and then build out another host for the 6 HDDs.

What I'm unsure about is how best to go about removing the datastore that the HDDs are tied to from this instance of PBS.
I'm thinking moving the HDDs to the new host with a fresh PBS install will be fine, I just do the PBS setup and then import the ZFS pool. Do let me know if there's a problem with that.
Now I'm not sure if there's anything 'special' I'd need to do on the existing PBS instance to disconnect the HDD datastore from the system. Can I just disconnect/remove it without the drives connected (and therefore with the datastore offline) or will that cause errors/issues which require me to first connect the datastore in order to disconnect/remove it?

I do have spare hardware as a test bench that I can use in order to boot the PBS instance with all 8 drives connected, if that's something I need to do.

Thanks for any replies I receive

I am unclear what you are meaning by 2 hosts. Do you intend to create 2 separate PBS nodes?

Easiest thing would be of course to replace the PSU. I am assuming it is not possible in your current scenario. If your original ZFS drives are intact from the PBS, you can simply transfer all the drives belonging to ZFS pool to the new host/node, import zpool, mount to the same mountpoint as before, add to PBS datastore using mount path. The PBS should pick up all your backup content from the mount point.

You could also copy the location /etc/proxmox-backup/ from old node to the new node. That location has all info related to your PBS and will save you time by not having to reconfigure your new PBS.You do not need to 'disconnect' current PBS datastore.

 

[snowturtle]

I am unclear what you are meaning by 2 hosts. Do you intend to create 2 separate PBS nodes?

Easiest thing would be of course to replace the PSU. I am assuming it is not possible in your current scenario. If your original ZFS drives are intact from the PBS, you can simply transfer all the drives belonging to ZFS pool to the new host/node, import zpool, mount to the same mountpoint as before, add to PBS datastore using mount path. The PBS should pick up all your backup content from the mount point.

You could also copy the location /etc/proxmox-backup/ from old node to the new node. That location has all info related to your PBS and will save you time by not having to reconfigure your new PBS.You do not need to 'disconnect' current PBS datastore.

I am unclear what you are meaning by 2 hosts. Do you intend to create 2 separate PBS nodes?

Okay so to provide some clarification: I have 3 servers in this situation:

• Server A is my old (now 'dead') PBS host that has 2 M.2 drives, 6 HDDs and then a 2.5" for the PBS OS.
• Server B is newish hardware that I want to make into my main PBS host. The issue with this is that the hardware only supports the two M.2's and the OS drive.
• Server C would be the host for the 6 HDDs.

The M.2's are a ZFS pool of Datastore A and the 6 HDDs are a ZFS pool for Datastore B

Easiest thing would be of course to replace the PSU. I am assuming it is not possible in your current scenario

It's unknown if it's possible. The old Server A hardware was on Dell Optiplex 790 that's 11 years old (kinda surprised the PSU lasted so long tbh). It's possible to get new PSU's but they'd all be second hand which seems like it could cause a fire risk so I've been hesitant to do that. I'm also pretty sure I overdid it on the old PSU because it was a 250W unit powering core PC parts plus SAS HBA, 2.5GB NIC, M.2 HBA, M.2's the,selves, 6 HDDS and the OS SSD.

If your original ZFS drives are intact from the PBS, you can simply transfer all the drives belonging to ZFS pool to the new host/node, import zpool, mount to the same mountpoint as before, add to PBS datastore using mount path. The PBS should pick up all your backup content from the mount point.

Yeah I'm understanding this will work for Server B because it will be getting the M.2's of Datastore A and the original OS disk, but Server C won't be getting that OS disk. Becuase it's ZFS I'm sure I can just get a new PBS OS for Server C and ZFS import eh pool (or something to that extent).
My question/concern really comes down to the fact that Datastore B still exists on the PBS OS on Server B, but that server can't be connected to the 6 HDDs and therefore it will report Datstore B as offline, I'm unsure if I can just remove/disconnect Datastore B from Server B whilst the datastore is unavailable/offline.

Does that all make a bit more sense, I know it's not the simplest situation.

 

[snowturtle]

You didn't used the SSDs as special devices or parts of these as metadata-only-L2ARC to boost the performance of the HDDs? This would have increased the GC performance by magnitudes and a little bit with backup/restore/verify.

No I didn't this is becuase these M.2's are either SATA M.2's or just really slow NVME. The old host hardware (as mentioned in my other reply) was an 11 year old Dell Optiplex 790 with (I think) PCIE 2, no M.2 support so those had to be connected via a carrier card / HBA and connected to the system via SATA ports. I had a feeling I'd be throttling both the M.2's and the HDDs if I were to use the M.2's as 'special' vdevs.

 

[Dunuin]

MSATA SSDs would still be like 500 times faster than your HDDs.
Everything is better than running a HDD-only datastore. Manual clearly recommends to best use SSD-only or in case you still want to use HDDs, combine HDD+SSD but not HDD-only.

And yes, it is possible to import the ZFS pool and reuse your existing datastore on it with a new PBS installation. But there is no feature to import existing datastores, so you will have to manually edit the /etc/proxmox-backup/datastore.cfg.
But again, not great to run HDD-only and buying another pair of SSDs for special devices would be a good idea.

 

[snowturtle]

MSATA SSDs would still be like 500 times faster than your HDDs.
Everything is better than running a HDD-only datastore. Manual clearly recommends to best use SSD-only or in case you still want to use HDDs, combine HDD+SSD but not HDD-only.

And yes, it is possible to import the ZFS pool and reuse your existing datastore on it with a new PBS installation. But there is no feature to import existing datastores, so you will have to manually edit the /etc/proxmox-backup/datastore.cfg.
But again, not great to run HDD-only and buying another pair of SSDs for special devices would be a good idea.

Thanks for the reply.

Okay, so trying to dig to the root of my question: Is there anything 'special' I need to do to remove/disconnect my HDD datastore from PBS (on Server B) so that it will not complain that the datastore is offline/unavailable? Will the datastore and drives need to be online/connected for it to edit the relevant configs (maybe it needs to look for those device ids existing) or should it just work and not throws any errors?

 

[Dunuin]

Before removing the disks you should enable the maintainance mode for that datastore and export the pool via "zpool export".

 

[snowturtle]

Before removing the disks you should enable the maintainance mode for that datastore and export the pool via "zpool export".

Datastore's in maintanece mode but the zpool cli command won't export the pool because "pool is busy", I think I can do zpool export -f <pool>and it should export but I'm not sure if PBS will get upset about that

 

[snowturtle]

Datastore's in maintanece mode but the zpool cli command won't export the pool because "pool is busy", I think I can do zpool export -f <pool>and it should export but I'm not sure if PBS will get upset about that

Okay found this thread which suggests trying the following, did that and now the zpool export has worked. Guess I go just go to the top right and hit "remove datastore" now?

systemctl restart proxmox-backup.service
systemctl restart proxmox-backup-proxy.service

Edit: Can't remember what I did exactly to get around the "pool is busy" error, I think it was just those systemctl commands and maybe a reboot. Once I was able to export the pool I needed to remove the mount in /mnt/datastore/<datastore to be removed> and then go to the "remove datastore" button (or maybe vice versa).