ZFS & AMD Ryzen/Threadripper?

[Norman Uittenbogaart]

I saw that QNAP has a NAS storage with a Ryzen processor.
Was wondering if anyone has a proxmox build with a Ryzen CPU and ZFS drives.
If so what is your configuration?

 

[Alwin]

Gigabyte AX370-Gaming K5-CF
AMD Ryzen 7 1800X Eight-Core Processor
4 x 16 GB 2133 MHz Kingston RAM
Samsung SSD 960 PRO 1TB (NVMe)
Crucial MX300 1TB (SSD)
HGST Ultrastar 7K6000 4TB

My development machine. All run on separate ZFS pools. Latest BIOS and firmware versions will be needed for stable operation. The memory I had worked stable after the last two AGESA updates.

 

[Norman Uittenbogaart]

My development machine. All run on separate ZFS pools. Latest BIOS and firmware versions will be needed for stable operation. The memory I had worked stable after the last two AGESA updates.

would you run production on it for a small company?
I figured if QNAP uses it in a NAS it must be stable....

 

[LnxBil]

would you run production on it for a small company?
I figured if QNAP uses it in a NAS it must be stable....

For SOHO, it could be find. I'd only use server grade hardware for a small company (small means for me <20)

 

[wolfgang]

The main Problem is about Ryzen/Threadripper as LnxBil meant you get only consumer Boards.
QNAP build there own MB.
Also, there is no ECC for Ryzen available.

 

[joshin]

So long as the MB supports it, ECC works fine on Ryzen/Threadripper. See: https://www.reddit.com/r/Amd/comments/6f7s28/what_is_the_state_of_ryzen_and_ecc/

I just wish there was a Threadripper MB with IPMI. I don't need the PCI-E lanes of Epyc, just raw CPU power.

The main Problem is about Ryzen/Threadripper as LnxBil meant you get only consumer Boards.
QNAP build there own MB.
Also, there is no ECC for Ryzen available.

 

[wolfgang]

So long as the MB supports it, ECC works fine on Ryzen/Threadripper. See:

Yes, I know but only the ASrock micro ATX boards seems to support ECC.
I board an Gigabyte K5 what should also support ECC but after a bios update, they disabled ECC.
I read that theoretically it work but the memory controller problems forced them to disable it.

 

[PLE]

Afaik many ASRock Ryzen Boards have ECC support enabled by default.

I doubt it has anything with the memory controller to do. ECC is even an official supported feature on the Ryzen Threadripper platform.

 

[Norman Uittenbogaart]

Afaik many ASRock Ryzen Boards have ECC support enabled by default.

I doubt it has anything with the memory controller to do. ECC is even an official supported feature on the Ryzen Threadripper platform.

I have a Ryzen 1600 in my home PC and I do know that the AMD processors are very "picky" on there memory support.
At least with running it on the full speed.
Not sure how this would work with ECC though.

 

[PLE]

Unbuffered ECC memory modules have just an additional chip for the ECC correction. They will run like regular memory in non-ECC boards.

I am using here Samsung DIMM 16GB, DDR4-2400, CL17-17-17, ECC (M391A2K43BB1-CRC) Modules in my Threadripper based development machine without any problem. The memory controller handles ECC support just fine:

Jul 26 13:22:52 threadripper kernel: [    0.144002] EDAC MC: Ver: 3.0.0
Jul 26 13:22:52 threadripper kernel: [    4.482056] EDAC amd64: Node 0: DRAM ECC enabled.

 

[LnxBil]

I don't need the PCI-E lanes of Epyc, just raw CPU power.

Those PCIe lanes are the killer argument for the use in a server environment. At the moment, you really need to populate all sockets in 1 HE box to get all possible expansion slots working.

 

[Norman Uittenbogaart]

Mm thinking a bit further, if you would have a Ryzen running.
It would have a max memory of 64GB.
If you then for example would have 48TB of zfs storage, would there be enough memory available to run VM's?

Maybe interesting here are two servers build with Ryzen,

and this one runs Proxmox

 

[LnxBil]

It would have a max memory of 64GB. If you then for example would have 48TB of zfs storage, would there be enough memory available to run VM's?

Depends on the number of VMs and how fast or slow your system has to operate.

For example:
I've been running ZFS on a Raspberry Pi with LXC on 512 MB of RAM and it was enough - it wasn't fast, but OK. Our main backup server uses 72 GB of RAM without VMs (not PVE on this one) with 48 TB pool and it is not enough for everything at once and we also have L2ARC in place.

 

[Norman Uittenbogaart]

One more thing I was wondering.
Would it make more sense to have one server with the storage and the VM's.
Or would it be more advisable to buy/build a NAS and run everything from there.

Don't think I will need to expand to more servers in the future.

 

[LnxBil]

Depends on the availability concerns. Two servers fail statistically more often than one.

I would not buy two servers, I'd use the money for the two to buy a better single server with redundancy in mind, ecc memory, more disk spindles, two psu's, etc.