nvme list + pveperf very low for nvme Samsung 980 PRO 1T

M

mpobieda

Member
Aug 3, 2022
17
0
6
root@dc1pve3:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 22323A78A00E CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 S5GXNX1TA00882V Samsung SSD 980 PRO 1TB 1 210.49 MB / 1.00 TB 512 B + 0 B 5B2QGXA7

The command output always returns Usage = x / y.
For disks:
1. KINGSTON SKC3000S1024G ( x = y = 1.02 TB )
2. CT1000P5PSSD8 ( x = y = 1.00 TB )
3. !!! Samsung SSD 980 PRO 1TB (x < y) !!!
Why x < y for Samsung 980 PRO drives?
Samsung 980 PRO drives have a very low pveperf (FSYNCS < 500).
pveperf for Kingston or Crucial gives satisfactory results (FSYNCS > 1500).

Examples:

root@dc1pve2:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 220635A6455B CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 50026B7685F0CD45 KINGSTON SKC3000S1024G 1 1.02 TB / 1.02 TB 512 B + 0 B EIFK31.6

root@dc1pve2:~# zpool status -vt nvme
pool: nvme
config:
NAME STATE READ WRITE CKSUM
nvme ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
nvme-CT1000P5PSSD8_220635A6455B ONLINE 0 0 0 (100% trimmed, completed at Sun 05 Nov 2023 12:31:43 AM CET)
nvme-KINGSTON_SKC3000S1024G_50026B7685F0CD45 ONLINE 0 0 0 (100% trimmed, completed at Sun 05 Nov 2023 12:31:43 AM CET)

root@dc1pve2:/etc# hdparm -tT /dev/nvme0n1
/dev/nvme0n1:
Timing cached reads: 39302 MB in 1.98 seconds = 19834.49 MB/sec
Timing buffered disk reads: 4692 MB in 3.00 seconds = 1563.32 MB/sec

root@dc1pve2:/etc# hdparm -tT /dev/nvme1n1
/dev/nvme1n1:
Timing cached reads: 43520 MB in 1.98 seconds = 21987.25 MB/sec
Timing buffered disk reads: 5014 MB in 3.00 seconds = 1670.63 MB/sec

root@dc1pve2:/etc# hdparm -tT --direct /dev/nvme0n1
/dev/nvme0n1:
Timing O_DIRECT cached reads: 1796 MB in 2.00 seconds = 897.80 MB/sec
Timing O_DIRECT disk reads: 5186 MB in 3.00 seconds = 1728.18 MB/sec

root@dc1pve2:/etc# hdparm -tT --direct /dev/nvme1n1
/dev/nvme1n1:
Timing O_DIRECT cached reads: 3802 MB in 2.00 seconds = 1902.19 MB/sec
Timing O_DIRECT disk reads: 5872 MB in 3.00 seconds = 1956.43 MB/sec

root@dc1pve2:/etc# pveperf /nvme
CPU BOGOMIPS: 27999.28
REGEX/SECOND: 4560086
HD SIZE: 524.28 GB (nvme)
FSYNCS/SECOND: 1966.73

root@dc1pve2:/etc# pveperf /nvme
CPU BOGOMIPS: 27999.28
REGEX/SECOND: 4753820
HD SIZE: 524.28 GB (nvme)
FSYNCS/SECOND: 2173.97

root@dc1pve2:/etc# pveperf /nvme
CPU BOGOMIPS: 27999.28
REGEX/SECOND: 4697025
HD SIZE: 524.28 GB (nvme)
FSYNCS/SECOND: 2102.16

######################################################################

root@dc1pve3:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 22323A78A00E CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 S5GXNX1TA00882V Samsung SSD 980 PRO 1TB 1 210.49 MB / 1.00 TB 512 B + 0 B 5B2QGXA7

root@dc1pve3:~# zpool status -vt nvme0n1
pool: nvme0n1
config:
NAME STATE READ WRITE CKSUM
nvme0n1 ONLINE 0 0 0
nvme0n1p1 ONLINE 0 0 0 (100% trimmed, completed at Wed 08 Nov 2023 01:42:25 PM CET)

root@dc1pve3:~# zpool status -vt nvme1n1
pool: nvme1n1
config:
NAME STATE READ WRITE CKSUM
nvme1n1 ONLINE 0 0 0
nvme-Samsung_SSD_980_PRO_1TB_S5GXNX1TA00882V ONLINE 0 0 0 (100% trimmed, completed at Wed 08 Nov 2023 01:42:23 PM CET)

root@dc1pve3:~# hdparm -tT /dev/nvme0n1
/dev/nvme0n1:
Timing cached reads: 31472 MB in 1.97 seconds = 15980.80 MB/sec
Timing buffered disk reads: 7048 MB in 3.00 seconds = 2349.28 MB/sec

root@dc1pve3:~# hdparm -tT /dev/nvme1n1
/dev/nvme1n1:
Timing cached reads: 31764 MB in 1.97 seconds = 16121.64 MB/sec
Timing buffered disk reads: 6592 MB in 3.00 seconds = 2197.01 MB/sec

root@dc1pve3:~# hdparm -tT --direct /dev/nvme0n1
/dev/nvme0n1:
Timing O_DIRECT cached reads: 3434 MB in 2.00 seconds = 1719.44 MB/sec
Timing O_DIRECT disk reads: 7762 MB in 3.00 seconds = 2586.91 MB/sec

root@dc1pve3:~# hdparm -tT --direct /dev/nvme1n1
/dev/nvme1n1:
Timing O_DIRECT cached reads: 4788 MB in 2.00 seconds = 2398.64 MB/sec
Timing O_DIRECT disk reads: 7682 MB in 3.00 seconds = 2560.21 MB/sec

root@dc1pve3:~# pveperf /nvme0n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 3635594
HD SIZE: 899.25 GB (nvme0n1)
FSYNCS/SECOND: 2252.60

root@dc1pve3:~# pveperf /nvme1n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 3422000
HD SIZE: 899.25 GB (nvme1n1)
FSYNCS/SECOND: 470.29

root@dc1pve3:~# pveperf /nvme0n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 3603209
HD SIZE: 899.25 GB (nvme0n1)
FSYNCS/SECOND: 2305.81

root@dc1pve3:~# pveperf /nvme1n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 3578267
HD SIZE: 899.25 GB (nvme1n1)
FSYNCS/SECOND: 463.40
 
Last edited:
mpobieda said:
For disks:
1. KINGSTON SKC3000S1024G ( x = y = 1.02 TB )
2. CT1000P5PSSD8 ( x = y = 1.00 TB )
3. !!! Samsung SSD 980 PRO 1TB (x < y) !!!
Why x < y for Samsung 980 PRO drives?
Samsung 980 PRO drives have a very low pveperf (FSYNCS < 500).
pveperf for Kingston or Crucial gives satisfactory results (FSYNCS > 1500).
Click to expand...
Those are consumer/prosumer drives and only fast on their marketing paper, not in real life.

Sorry, but I did not find any number there satisfactory. My 10y old Samsung Enterprise SATA drives perform better on a lowest end Intel N6005 with 3000 fsyncs and a very slow sata bridge.
 
LnxBil said:
Those are consumer/prosumer drives and only fast on their marketing paper, not in real life.

Sorry, but I did not find any number there satisfactory. My 10y old Samsung Enterprise SATA drives perform better on a lowest end Intel N6005 with 3000 fsyncs and a very slow sata bridge.
Click to expand...
Do you know why Samsung has such a value of 210.49 MB / 1.00 TB? And not like 1.00 TB / 1.00 TB?

I am going to send this disk for warranty that it does not meet my performance expectations in relation to the drives I have from Kingston and Crucial. Before sending it to the service center, I would like to be sure that the Samsung drive works less efficiently on the Proxmox platform than the Kingson and Crucial drives.

Kingston and Crucial drives are also consumer grade, but they achieve 4x greater FSYNCS performance. My test platform is DELL T-40 and DELL T-30. I don't expect great performance because these are test/study servers.
 
LnxBil said:
Have you tried analyzing the SSD with this samsung magician?
Click to expand...
Yes, I tried. It says it's ok, just like hdparam. But pveperf reports lower performance. I have three identical Samsung 980 pro drives and according to pveperf, all of them are worse than Kingsone and Crucial.
 
mpobieda said:
Yes, I tried. It says it's ok, just like hdparam. But pveperf reports lower performance. I have three identical Samsung 980 pro drives and according to pveperf, all of them are worse than Kingsone and Crucial.
Click to expand...
It tests similar to my Samsung 970 EVO (about 200, where my enterprise drives reach 17500). Could just be the implementation of Samsung's controller (and firmware). Other brands use other controllers.
 
leesteken said:
It tests similar to my Samsung 970 EVO (about 200, where my enterprise drives reach 17500). Could just be the implementation of Samsung's controller (and firmware). Other brands use other controllers.
Click to expand...
Ok thanks.

Do you know why Samsung has such a value of 210.49 MB / 1.00 TB? And not such 1.00 TB / 1.00 TB?

root@dc1pve3:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 22323A78A00E CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 S5GXNX1TA00882V Samsung SSD 980 PRO 1TB 1 210.49 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
 
LnxBil said:
pveperf is good to get a rough estimate, have you tried fio to get more comparable numbers?
Click to expand...
I've never used fio.
I don't need detailed analysis.
I focus on comparing drives of different brands using the same method.
The differences in pveperf results are huge for the Samsung 980 PRO drive, compared to Kingstone and Crucial.
Comparing pveperf results between Kingstone and Crucial yields similar results.

For now, the comparison method in pveperf results is sufficient for me.
I will simply avoid company Samsung in the future.

I'm just wondering why for Samsung I get a result of 210.49MB / 1TB as a result of the "nvme list" command.

root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 212.36 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 212.41 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403

root@dc1pve3:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 22323A78A00E CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 S5GXNX1TA00882V Samsung SSD 980 PRO 1TB 1 210.49 MB / 1.00 TB 512 B + 0 B 5B2QGXA7

root@dc1pve1:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 50026B7685F0CC27 KINGSTON SKC3000S1024G 1 1.02 TB / 1.02 TB 512 B + 0 B EIFK31.6
/dev/nvme1n1 50026B7382673E1B KINGSTON SKC3000S1024G 1 1.02 TB / 1.02 TB 512 B + 0 B EIFK31.6

root@dc1pve2:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 220635A6455B CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 50026B7685F0CD45 KINGSTON SKC3000S1024G 1 1.02 TB / 1.02 TB 512 B + 0 B EIFK31.6
 
Last edited:
I have detected that the USAGE value in Samsung drives changes after TRIM is performed.

Before trim:

root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 212.41 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 212.36 MB / 1.00 TB 512 B + 0 B 5B2QGXA7

After trim:

root@pve:~# zpool trim nvme0n1
root@pve:~# zpool trim nvme1n1
root@pve:~# zpool trim nvme2n1
root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 11.20 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 11.30 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
 
This Samsung 980 pro has to do something in addition to Kingstone and Crucial. As if it was doing some indexing for TRIM at the moment of the PVEPERF test???

Sorry for my English, it's the result of google translate.

root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 11.30 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 11.20 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403

#############################

root@pve:~# pveperf /nvme0n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4179171
HD SIZE: 899.25 GB (nvme0n1)
FSYNCS/SECOND: 1959.83

root@pve:~# pveperf /nvme1n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4169668
HD SIZE: 899.25 GB (nvme1n1)
FSYNCS/SECOND: 455.83

root@pve:~# pveperf /nvme2n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4057095
HD SIZE: 899.25 GB (nvme2n1)
FSYNCS/SECOND: 430.13

#############################

root@pve:~# pveperf /nvme0n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4149994
HD SIZE: 899.25 GB (nvme0n1)
FSYNCS/SECOND: 1793.94

root@pve:~# pveperf /nvme1n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4199623
HD SIZE: 899.25 GB (nvme1n1)
FSYNCS/SECOND: 437.74

root@pve:~# pveperf /nvme2n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4100486
HD SIZE: 899.25 GB (nvme2n1)
FSYNCS/SECOND: 442.33

root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 22.11 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 22.32 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403

#############################

root@pve:~# pveperf /nvme0n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4144190
HD SIZE: 899.25 GB (nvme0n1)
FSYNCS/SECOND: 2019.62

root@pve:~# pveperf /nvme1n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 3999872
HD SIZE: 899.25 GB (nvme1n1)
FSYNCS/SECOND: 444.88

root@pve:~# pveperf /nvme2n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4170076
HD SIZE: 899.25 GB (nvme2n1)
FSYNCS/SECOND: 449.36

root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 27.72 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 27.88 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403

#############################

!!!!!!!!!!!!!!!

root@pve:~# pveperf /nvme2n1
CPU BOGOMIPS: 26399.92
REGEX/SECOND: 4213400
HD SIZE: 899.25 GB (nvme2n1)
FSYNCS/SECOND: 440.84

root@pve:~# nvme list
Node Generic SN Model Namespace Usage Format FW Rev
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme2n1 /dev/ng2n1 S5GXNX1TA00929R Samsung SSD 980 PRO 1TB 1 33.25 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme1n1 /dev/ng1n1 S5GXNX0TB07492M Samsung SSD 980 PRO 1TB 1 27.88 MB / 1.00 TB 512 B + 0 B 5B2QGXA7
/dev/nvme0n1 /dev/ng0n1 220635A63F31 CT1000P5PSSD8 1 1.00 TB / 1.00 TB 512 B + 0 B P7CR403

!!!!!!!!!!!!!!!

Or maybe it indexes the saved data while writing to make reading faster?

!!!!!!!!!!!!!!!

root@pve:~# hdparm -tT /dev/nvme0n1
/dev/nvme0n1:
Timing cached reads: 33726 MB in 1.97 seconds = 17133.28 MB/sec
Timing buffered disk reads: 5100 MB in 3.00 seconds = 1699.53 MB/sec

root@pve:~# hdparm -tT /dev/nvme1n1
/dev/nvme1n1:
Timing cached reads: 34294 MB in 1.97 seconds = 17425.77 MB/sec
Timing buffered disk reads: 8764 MB in 3.00 seconds = 2921.08 MB/sec

root@pve:~# hdparm -tT /dev/nvme2n1
/dev/nvme2n1:
Timing cached reads: 34114 MB in 1.97 seconds = 17333.87 MB/sec
Timing buffered disk reads: 8206 MB in 3.00 seconds = 2734.95 MB/sec

#############################

root@pve:~# hdparm -tT --direct /dev/nvme0n1
/dev/nvme0n1:
Timing O_DIRECT cached reads: 2362 MB in 2.00 seconds = 1181.92 MB/sec
Timing O_DIRECT disk reads: 7492 MB in 3.00 seconds = 2497.13 MB/sec

root@pve:~# hdparm -tT --direct /dev/nvme1n1
/dev/nvme1n1:
Timing O_DIRECT cached reads: 5218 MB in 2.00 seconds = 2614.37 MB/sec
Timing O_DIRECT disk reads: 7898 MB in 3.00 seconds = 2632.36 MB/sec

root@pve:~# hdparm -tT --direct /dev/nvme2n1
/dev/nvme2n1:
Timing O_DIRECT cached reads: 4858 MB in 2.00 seconds = 2434.25 MB/sec
Timing O_DIRECT disk reads: 7426 MB in 3.00 seconds = 2474.98 MB/sec
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom