Metadata cache on ZFS - How much does it help for PBS?

[mgaudette112]

Hi,

Question about storage support for PBS : this is from the PBS web site, under recommended specs . (bold is mine for emphasis)

• Backup storage:
  • Use only SSDs, for best results
  • If HDDs are used: Using a metadata cache is highly recommended, for example, add a ZFS special device mirror.

I have, on another server that needed to backup about 6TB of data, tried both HDD by themselves and SSD. HDD were ok but almost too slow to do daily backups (verify jobs took 12 hours). I figured I could shell out the money for SSDs, which I did.

I have another PBS to figure out, this time backing up about 20TB of data. I am wondering, if I was to use ZFS with special vdev, as "recommended" above, am I getting close-to-SSD performance or is it barely better than HDD?

I am trying to figure out where HDD+special vdev lies in the performance spectrum compared to SSD-only and HDD-only.

 

[Chris]

Hi,

Question about storage support for PBS : this is from the PBS web site, under recommended specs . (bold is mine for emphasis)


I have, on another server that needed to backup about 6TB of data, tried both HDD by themselves and SSD. HDD were ok but almost too slow to do daily backups (verify jobs took 12 hours). I figured I could shell out the money for SSDs, which I did.

I have another PBS to figure out, this time backing up about 20TB of data. I am wondering, if I was to use ZFS with special vdev, as "recommended" above, am I getting close-to-SSD performance or is it barely better than HDD?

I am trying to figure out where HDD+special vdev lies in the performance spectrum compared to SSD-only and HDD-only.

Hi,
the metadata cache will help improve the performance for operations where the PBS server only needs to access the file metadata such as the atime in case of garbage collection. For verify jobs, which need to read and hash the data stored with the chunk store, the metadata cache will help not that much, you will not get the same performace as for an ssd only storage for these operations.

 

[mgaudette112]

Hi,
the metadata cache will help improve the performance for operations where the PBS server only needs to access the file metadata such as the atime in case of garbage collection. For verify jobs, which need to read and hash the data stored with the chunk store, the metadata cache will help not that much, you will not get the same performace as for an ssd only storage for these operations.

Thank you for the clarification