ZFS Special Device question

[Dunuin]

Hi,

Can someone tell me what will happen when you add a special device (only for metadata) to a pool and that special device gets full? Will the pool get into troubles because it can't write any new metadata or will it just write new metadata to the other non special device disks in that pool?

I asked because I got 8x 8TB HDDs and 2x 200GB SSDs and rule of thumb is that metadata will need around 0.3% of the pool size. In my case that would be 192GB so there is not much capacity left.

 

[ph0x]

Will you use them in JBOD mode? Because if not then there will already be a lot less usable space and with that less need for metadata.
From my understanding it's 0.3% of the usable space.

 

[Dunuin]

The idea was to use 8x 8TB as raidz2 for data (so 48TB usable space and with quota of 90% set so only 43,2TB usable) and 2x 200GB as mirror (maybe 3x 200GB later) for metadata. So 0.3% of 43,2TB would be only 130GB.

 

[ph0x]

I would deem that appropriate but it's still pretty close and maybe someone has an answer with more insight.
That's just the way I understood it ...

 

[ph0x]

This post says it would spill over onto the HDDs if the Special Device became full.

https://www.reddit.com/r/zfs/comments/cm594b/why_is_nobody_talking_about_the_newly_introduced/

 

[jasonsansone]

It will spill over. You can add a metadata device to a pool at anytime, but only newly created files will have their metadata stored on the special device. Existing files will remain entirely on the prior existing vdev. If a pool exists with sample data, it’s very easily to run commands to check how much metadata is used. I can find the link if you need instructions. I made some posts on the TrueNAS forum about this subject.