[Share] Some tweaks for Threadripper home server

Discussion in 'Proxmox VE: Installation and configuration' started by MasterPhi, Jan 11, 2019.

  1. MasterPhi

    MasterPhi New Member

    Joined:
    Jan 7, 2019
    Messages:
    3
    Likes Received:
    1
    Hi,

    I built a home server without difficulty thanks to Proxmox :

    CPU: Threadripper 1920X
    RAM: 64GB ECC
    MB: Asrock X399 Professional Gaming @ BIOS 3.30
    GPU: Inno3D 1050Ti
    HBA: SAS9305-16i
    ZFS Pool: HGST He10 10TB * 3 + WD Red 3TB *3s

    GPU and USB passthrough works very well, since there are many tutorials available I'm not going to detail here.

    However, there are several small things which makes the configuration not perfect.

    So I made some tweaking(hacking) scripts. ;)

    Code:
    github.com/HiFiPhile/PVE_Tweaks
    CPU pinning
    Even if NUMA is specified in the VM configuration file, since there is no fixed mapping between vCPUs and Host CPUs, cross-node access can still be happened. In my case it reflects in Aida64 memory benchmark with random speed drop down (from 46GB/s to 35GB/s) and latency increse (form 86ns to 140ns).

    This script by ayufan handles the cpu pinning very well.
    Code:
    gist.github.com/ayufan/37be5c0b8fd26113a8be
    I made some adjustments so you can specify which cores are used explicitly.
    Example: assign core 0-3 to VM100:
    Code:
    pve_cpu_pinning.sh 100 0,1,2,3
    Change CPU scheduler to FIFO
    Multiple threads indicate SCHD_FIFO can reduce latency (lag spikes).
    Code:
    pve_schd_fifo.sh 100
    Run custom command after VM start up
    I hacked QemuServer to run these tweaks at startup.:) (Use at your own risk !)
    Code:
    patch /usr/share/perl5/PVE/QemuServer.pm ./QemuServer_PostCommand.patch
    Reboot, then in your VM config file add:
    Code:
    postcommand: /home/your_tweaks.sh
    Disclaimer
    These tweaks are provided as-are, and there are no guarantees that they fit your purposes or that they are bug-free. Use it at your own risk!
     
    #1 MasterPhi, Jan 11, 2019
    Last edited: Jan 11, 2019
    nightcom likes this.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice