ZFS & AMD Ryzen/Threadripper?

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
2,816
247
63
Gigabyte AX370-Gaming K5-CF
AMD Ryzen 7 1800X Eight-Core Processor
4 x 16 GB 2133 MHz Kingston RAM
Samsung SSD 960 PRO 1TB (NVMe)
Crucial MX300 1TB (SSD)
HGST Ultrastar 7K6000 4TB
My development machine. All run on separate ZFS pools. Latest BIOS and firmware versions will be needed for stable operation. The memory I had worked stable after the last two AGESA updates.
 
  • Like
Reactions: chrone

wolfgang

Proxmox Staff Member
Staff member
Oct 1, 2014
4,966
331
83
The main Problem is about Ryzen/Threadripper as LnxBil meant you get only consumer Boards.
QNAP build there own MB.
Also, there is no ECC for Ryzen available.
 
  • Like
Reactions: chrone
Jul 23, 2013
101
8
18
Phoenix, Arizona

wolfgang

Proxmox Staff Member
Staff member
Oct 1, 2014
4,966
331
83
So long as the MB supports it, ECC works fine on Ryzen/Threadripper. See:
Yes, I know but only the ASrock micro ATX boards seems to support ECC.
I board an Gigabyte K5 what should also support ECC but after a bios update, they disabled ECC.
I read that theoretically it work but the memory controller problems forced them to disable it.
 

PLE

New Member
Aug 17, 2010
7
0
1
Afaik many ASRock Ryzen Boards have ECC support enabled by default.

I doubt it has anything with the memory controller to do. ECC is even an official supported feature on the Ryzen Threadripper platform.
 
Afaik many ASRock Ryzen Boards have ECC support enabled by default.

I doubt it has anything with the memory controller to do. ECC is even an official supported feature on the Ryzen Threadripper platform.
I have a Ryzen 1600 in my home PC and I do know that the AMD processors are very "picky" on there memory support.
At least with running it on the full speed.
Not sure how this would work with ECC though.
 

PLE

New Member
Aug 17, 2010
7
0
1
Unbuffered ECC memory modules have just an additional chip for the ECC correction. They will run like regular memory in non-ECC boards.

I am using here Samsung DIMM 16GB, DDR4-2400, CL17-17-17, ECC (M391A2K43BB1-CRC) Modules in my Threadripper based development machine without any problem. The memory controller handles ECC support just fine:
Code:
Jul 26 13:22:52 threadripper kernel: [    0.144002] EDAC MC: Ver: 3.0.0
Jul 26 13:22:52 threadripper kernel: [    4.482056] EDAC amd64: Node 0: DRAM ECC enabled.
 

LnxBil

Well-Known Member
Feb 21, 2015
4,231
412
83
Germany
I don't need the PCI-E lanes of Epyc, just raw CPU power.
Those PCIe lanes are the killer argument for the use in a server environment. At the moment, you really need to populate all sockets in 1 HE box to get all possible expansion slots working.
 
Mm thinking a bit further, if you would have a Ryzen running.
It would have a max memory of 64GB.
If you then for example would have 48TB of zfs storage, would there be enough memory available to run VM's?

Maybe interesting here are two servers build with Ryzen,

and this one runs Proxmox

 

LnxBil

Well-Known Member
Feb 21, 2015
4,231
412
83
Germany
It would have a max memory of 64GB. If you then for example would have 48TB of zfs storage, would there be enough memory available to run VM's?
Depends on the number of VMs and how fast or slow your system has to operate.

For example:
I've been running ZFS on a Raspberry Pi with LXC on 512 MB of RAM and it was enough - it wasn't fast, but OK. Our main backup server uses 72 GB of RAM without VMs (not PVE on this one) with 48 TB pool and it is not enough for everything at once and we also have L2ARC in place.
 

LnxBil

Well-Known Member
Feb 21, 2015
4,231
412
83
Germany
Depends on the availability concerns. Two servers fail statistically more often than one.

I would not buy two servers, I'd use the money for the two to buy a better single server with redundancy in mind, ecc memory, more disk spindles, two psu's, etc.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!