Diagnosing cause of high RAM latency

Terminatrix

New Member
Aug 23, 2021
18
1
3
31
I have two identical nodes however I am having trouble pinning down why one node seems to perform a lot worse than the other. I believe it maybe due to these screenshots below. Node1 seems to have more tasks and threads running. I am running benchmark tests to achieve 100% CPU usage on node2 while the loads are similar the performance and latency on the RAM is vastly different.

Node 1:
225ef05decd6479bb9e5d099950c95fe.png

Node 2:
d44e6c24f23c3fb7131d3bed07e78384.png


Node1 sometimes drops below 1000MiB/sec with 5000+ max latency

Code:
sysbench --test=memory --memory-block-size=4G --memory-total-size=32G run

Node1:
Code:
Threads started!


Total operations: 6 (    0.59 per second)


24576.00 MiB transferred (2408.07 MiB/sec)




General statistics:
    total time:                          10.2044s
    total number of events:              6


Latency (ms):
         min:                                 1118.89
         avg:                                 1700.65
         max:                                 3116.47
         95th percentile:                     3095.38
         sum:                                10203.93


Threads fairness:
    events (avg/stddev):           6.0000/0.00
    execution time (avg/stddev):   10.2039/0.00

Node2:

Code:
Threads started!


Total operations: 8 (    1.99 per second)


32768.00 MiB transferred (8141.01 MiB/sec)




General statistics:
    total time:                          4.0237s
    total number of events:              8


Latency (ms):
         min:                                  462.73
         avg:                                  502.92
         max:                                  524.65
         95th percentile:                      520.62
         sum:                                 4023.39


Threads fairness:
    events (avg/stddev):           8.0000/0.00
    execution time (avg/stddev):   4.0234/0.00

What do you suggest I run to give us a better idea?