Moving old backups to HDD

[jw8]

Hello all,

I was hoping someone could point me in the right direction on how to do this, but basically I had the following setup:

1x 1 tb nvme drive - for boot
2x 4 tb Samsung SSD
2x 12 tb WD Red Pro HDD

I wanted to mirror the two SSD's together and mirror the two HDDs together. However, is there a way to store X number/time frame of backups on SSD and move the older ones from SSD to HDD, then prune the still older ones from the HDD? I'm trying to maximize the speed of the latest backups. I also see something about a special ZFS device - is that something that could help?

Thank you!

 

[UdoB]

However, is there a way to store X number/time frame of backups on SSD and move the older ones from SSD to HDD, then prune the still older ones from the HDD? I'm trying to maximize the speed of the latest backups. I also see something about a special ZFS device

You can have only one solution: either build two independent pools or have a Special Device. For rotating rust I do really, really, really recommend having a Special Device.

What I would not do is to install the OS on a non-redundant device. This leads to my personally preferred construct for the given hardware:

• install PBS on the HDDs, mirrored - as I am not aware of the possibility to add a Special Device during installation
• when everything works fine add the SSDs as a Special Device, mirrored; this requires some CLI work, I believe

The single NVMe stays unused in this picture. You may think to add it as Cache Device, but for a backup system most data is only read once and never twice - be it during write, during verify or during restore - a Cache does not help with this at all. And meta-data is stored in ARC, not in second level cache.

You may also think to partition the SSDs to reach your goal of implementing both a separate pool and a Special Device. While this is technically possible I do not recommended it. Only if this is a secondary PBS I would possibly go that route.

Anyway: when you have two independent pools to create two independent storages for that single PBS you can setup "Sync-Jobs" to copy a subset of the fast pool to the slow pool. Both have their own unique and independent retention rules. This way you can have a few backups on the fast pool and a very long history on the slow pool.

Again: your-mileage-may-vary drastically...

 

[news]

Thanks @UdoB this is the way!

What are these 4 tb Samsung SSD and 1 tb nvme drive?

I would setup some portions on these 4 tb Samsung SSD, if they are ZFS usable.

• Partition 1 and Part 2 are for the boot system, bios an uefi,
• on Partition 3 is rpool, this is the Debian 13/ Proxmox System. 250 GB as VDEV ZFS mirror is right ,
• Partition 4, with 16 GB can hold ZFS ZIL/LOG as VDEV ZFS mirror for your HDD ZFS pool dpool,
• Partition 5, with 610 GB (aprox 5% of your 12 TB HDD) can hold ZFS special device as VDEV ZFS mirror for your HDD ZFS pool dpool
• Partition 6, ex. 500 GB each, can not must use as ZFS cache drives of VDEV ZFS stripe with the two partiton 6.
  So you became 1 GB 2x the size from Partition 6. Here save the zfs system all the files, you most read. Can good for lxc or vm to start as fast it can be.
• Partition 7, can be the Rest, apx 2700 GB for a second fast Pool as VDEV ZFS mirror.

The nvme slot can be spare or for other use case. Some Mainboard have a PCIe x4 Slot, so you can setup a NVMe to PCIe x4 card for a 2nd NVMe.

 

[UdoB]

Thanks @UdoB this is the way!

Well..., you are using a lot of partition magic - basically that's what I do not like to recommend.

I know it works and I did do that too sometimes. But it complicates things massively. Especially for a "normal user": when a physical device fails it is already complicated enough to replace a device without handling multiple partitions and pools/vdevs...

But yeah... anything goes

 

[jw8]

I have set the two 12 TB HDDS in RAID1 and installed the OS to that device, and I have added the two 1 TB SSDs as a mirrored special device for that ZFS pool. However, I cannot seem to add the zfs pool (named rpool) to the datastore on the web gui. I have tried several different ways but cant seem to get it to work. Can you guys point me in the right direction on what to put in to the Add Datastore window?

When I try to add it and specify the backing path as /rpool, it says "datastore path not empty."

Any help is much appreciated. Thank you!

 

[jw8]

Nevermind, I was pointing it to /rpool and not /rpool/ROOT/pbs-1 for the backing directory (per the zfs list command)

Hopefully that helps someone else. Thank you guys for all the help!

 

[UdoB]

Nevermind,

Great that it works now

Please tag this thread "Solved" - there is an "Edit thread"-button above the first post.