Best ZFS storage layout for multiple clients

guerro

Member
Oct 1, 2021
12
0
21
23
Hello Everyone,
First i'd like to thank you for the amazing product and integration PVE and PBS have.

I work in a pretty small office who manages 20ish PVE all from different clients.
Lately i've been working on a "cloud" solution for our partners where they can upload all of their backup on our server through a VPN in order to have a geographic redoundancy. So far so great, my fast arranged setup is working great but we think is time to invest in something more planned out and dedicated.

At the moment we have 8x4tb drives in RAID-Z2 configuration on a TrueNAS machine. From there, an iSCSI connection is mounted on the PBS and a datastore is created for each of our clients. This means that throughout the day the PBS does multiple prunes and GC heavily impacting the overall performance of the drives.

We've now bought a dedicated machine with 16x4TB drives and i'm struggling to what configuration should i use...:
- Option A is to have a single big datastore across 2x8disk RAID10 with ZFS. Every client would have a private Namespace so they cannot interfere with eachother but I will have a much better deduplication. Also i would run a single GC.
- Option B is to have multiple datastores across 2x8disk RAID10 with ZFS. By doing this way i could manage how much space each client could take but with a much less deduplication efficency and with multiple GC running at the same time.

One thing I didn't yet tested is if I can have 2 different VM with the same VMID on 2 separate namespace.

What would you guys reccommend?
Sorry for the bad explanation and grammar, english is not my primary language.

Any effort is apprecciated!

Have a nice day.
 
GC really isn't a problem anymore after you add 2 or 3 enterprise SSDs as a mirrored special metadata device. This will increase GC performance by magnitudes and also doubles the performance of the verify/restore/backup tasks. So I would highly recommend to add some SSDs. But keep in mind that all data on the HDDs is lost as soon as you loose those SSDs, as this is no caching.
 
  • Like
Reactions: guerro
GC really isn't a problem anymore after you add 2 or 3 enterprise SSDs as a mirrored special metadata device. This will increase GC performance by magnitudes and also doubles the performance of the verify/restore/backup tasks. So I would highly recommend to add some SSDs. But keep in mind that all data on the HDDs is lost as soon as you loose those SSDs, as this is no caching.
Hi Dunuin,
Thank you for the fast reply.

At the moment I have 2x256GB SSD in the test machine.

How much metadata space do you think I would need with 64TB total raw space? I read that the metadata rateo is 34:1, do you think this is accurate?