Install Report - Supermicro 825TQC-R740LPB with H11SSL-NC MB Epyc with Samsung 860 Pro

[stompro]

Hello, no problems just wanted to report on my recent install.

• System from serversdirect - Around $3800 shipped (drives not included).
• AMD Epyc 7402P 24 core
• 128GB ram
• No drives
• Purchased 4 Samsung 860 Pro 2T drives separately - my disk needs are modest and I wanted to try to go cheap.
  • Setup as ZFZ Raid 10
• Purchased 1 Samsung 970 Pro 1T M.2 NVME separately.

For my first test install, I left the Broadcom 3008 raid card set to IR mode, but didn't configure any raid. That seemed to work fine, but I had boot up problems with the timing, and had to add various delays to the ZFS and kernel bootup.

I then saw the suggestion to never use raid controllers with ZFS, so I found a good guide on how to flash to IT mode at servethehome forms (LSI3008 flash it mode).

After that, the install seemed to go fine, but the system wouldn't boot normally, it would always come up with a grub rescue prompt with an error of "Unknown device 1". After poking at that a little while, I saw a suggestion to use UEFI boot instead of legacy. So I went into the H11SSL-NC bios and changed the boot mode from 'Both' (Legacy and UEFI) to just UEFI. Then I reinstalled again and the Proxmox installer figured out how to setup the correct boot mode, and now it boots without trouble. Even after the initial kernel upgrade.

Performance

I used the proxmox default ZFZ settings for the 860 Pro raid 10 - ashift 12, compression on. Setup trim 'zpool trim rpool' and 'zpool set autotrim=yes rpool' .

root@proxmox6:~# pveperf
CPU BOGOMIPS:      268767.36
REGEX/SECOND:      3105966
HD SIZE:           3688.99 GB (rpool/ROOT/pve-1)
FSYNCS/SECOND:     708.88
DNS EXT:           39.51 ms
DNS INT:           16.36 ms (larl.org)

Which seems fine from what I can find for these consumer drives.

Bonnie++ Results - bonnie++ -s258000 -n150 -u0

Version  1.98       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
proxmox6    258000M  272k  99  1.4g  95  1.1g  97  756k  99  2.5g  99 +++++ +++
Latency             27580us   25866us     100ms   10926us     726us    4230us
Version  1.98       ------Sequential Create------ --------Random Create--------
proxmox6            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                150 153600  97 +++++ +++ 153600  76 153600  99 +++++ +++ 153600  91
Latency             11433us     663us     481ms    6080us     226us   16719us

I may purchase better drives down the road if these seem to not have enough performance for my use.

I haven't decided how to use the Samsung 970 pro drive yet.

 

[mcflym]

Nice to read! I'm about to install a H11SSL-NC too. My CPU will be an AMD Epyc 7282 with 16 cores.