Best configuration for USB-HDDs used as NAS storage?

[dakazze]

Hello!
I am pretty new to all of this and already did my research but I am still not sure what config would be best/most stable considering the HW I have available:

I am using a pretty powerful Thin-Client with an i7 but since there is no way to add a PCI Raid controller I got myself a external USB 3.0 enclosure which includes a RAID controller. This enclosure takes 2 HDDs which I want to use in a RAID/mirrored configuration.
(usage: very important private and business data that is backed up to the cloud too for redundancy AND random backups, pictures, less important business stuff with no additional cloud backup)

Now I just need to decide if I should use the enclosures RAID or if I should use ZFS!
I am aware that using another PC as my home server would be optimal but since I already have the HW and it more than delivers enough power to run all of my stuff at low power consumption I need to make this work as good as possible.

In my research I gathered that HW raid in enclosures like this is not really reliable
AND
that it is better to use ZFS for this purpose
AND
that ZFS for USB-HDDs is not recommended

So I hope you understand my predicament and are willing to help (noob friendly).

 

[Dunuin]

In my research I gathered that HW raid in enclosures like this is not really reliable
AND
that it is better to use ZFS for this purpose
AND
that ZFS for USB-HDDs is not recommended

AND
that ZFS shouldn`t be used with a raid controller. See here: https://openzfs.github.io/openzfs-docs/Performance and Tuning/Hardware.html#hardware-raid-controllers
AND
that ZFS shouldn't be used with SMR HDDs (so make sure to get proper 24/7 rated NAS/Enterprise HDDs using CMR like a WD Red Plus/ WD Red Pro/Seagate Ironwolf...normal WD Reds for example use SMR and should be avoided)

 

[dakazze]

Ty!
So I guess it is best to just use the USB enclosures RAID 1 function?!

 

[Dunuin]

Or to not get a raid enclosure in the first place and use two single disk enclosures when ZFS is wanted because you care about your data and want stuff like bit rot protection.
But USB disks are always a bit problematic:
- often crappy disks inside and you never know that disk model is really used unless you buy a bulk disk and an empty enclosure
- USB is not the most stable protocol
- its external and vibrations are really bad for HDDs. So you can't stack them and you will always have to fear tripping over one of the wires head-crashing your disks
- the USB-to-SATA controllers of enclosures are usually pretty crappy
- USB enclosures are most of the time only passively cooled and when running them 24/7 the disks will get too hot increasing wear

 

[dakazze]

Thanks again!
I guess I will go for another solution when I find the time then.
Until then I will use the 2 disk usb enclosure at RAID 1 and do another BU in the meantime.

What would be the best practice to present this storage to my TrueNAS VM if it is the only machine that needs access?
USB device -> VM ?
(cant PCI pass the USB controller because I need the ports for other machines)

 

[Dunuin]

cant PCI pass the USB controller because I need the ports for other machines

USB passthough is slow and not that reliable. So probably normal disk passthough: https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM)

But keep in mind that your TrueNAS is then still working with virtual disks as you are not passing through real physical hardware. And TrueNAS only uses ZFS and ZFS shouldn't be used on top of HW raid. So The same problem again.

 

[dakazze]

USB passthough is slow and not that reliable. So probably normal disk passthough: https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM)

But keep in mind that your TrueNAS is then still working with virtual disks as you are not passing through real physical hardware. And TrueNAS only uses ZFS and ZFS shouldn't be used on top of HW raid. So The same problem again.

OK you got me, I´ll throw out the raid enclosure and get 2 singles.
So I create ZFS storage on proxmox and pass that to truenas?

 

[dakazze]

meh and again, searching the web says that zfs doesnt work well with usb devices....
Should I not just format the raid enclosure to NTFS and present that to truenas?

 

[dakazze]

Hmmmmmm now that I have the seperate enclusures I am facing something new:

TrueNAS is not getting SMART data from the disks
In Proxmox all SMART data is shown and looks sane
It doesnt make a difference if I pass the USB-Port/Device to the VM or if I pass the full USB controller to the VM

 

[Dunuin]

TrueNAS is not getting SMART data from the disks

Yes, because it only sees virtual disks. Disk passthrough isn't passing through the real disks. That will only work by PCI passthrough a whole USB controller where you probably would need to buy a PCIe USB card first (because the onboard USB controllers are usually in the same IOMMU group with the other onboard stuff).

 

[dakazze]

I was passing the USB controller though

 

[Dunuin]

I was passing the USB controller though

Using PCI passthrough?

 

[dakazze]

Yep - I passed the chipstes USB xHCI controller, which worked except SMART

Now I am facing still another issue..... When creating a new pool in TrueNAS, using the disks (both show the real serial number) it fails
"because the disks dont have unique serial numbers" which is just wrong! Even truenas sees the unique original serial numbers...