What is considered unusually high disk io for an NVME SSD ?

[rfox]

Just wondering - I have a small cluster of 5 nodes running various stuff - one node is an R86s (N5105) device running OPNSense as router - been working fine for many months now . . . but I noticed a problem recently which turned out to be the netflow monitor going nuts - I was experiencing extremely high disk io writes up to 80M. After resetting the netflow, things settled down again - but now I am averaging around 5M - I think because I use ZenArmor and NTopNG which does a bunch of statistics and logging

My question is: What is considered "normal" when it comes to io writes and when should I worry about nvme ssd excessive wear ? I don't necessarily need the NTopNG running constantly, which when turned off reduces the normal io writes down to 1.5M on average

Any best practice tips would be welcome!!

Here was the anomoly which I corrected by resetting the netflow data:


and here is the average now with NTopNG running:

 

[UdoB]

I was experiencing extremely high disk io writes up to 80M.

This is not much. Rotating rust allows up to 200 MB/s (or more) nowadays. NVMe can write data with a speed of a few Gigabytes per second. This really, really depends on the specific hardware - and "consumer grade" vs. "enterprise class" makes a huge difference.

and when should I worry about nvme ssd excessive wear ?

Look at the SMART data: web gui - <node> - Disks - <select disk> - top area button "Show SMART values".

It should give you some hints, but not all devices give the same structure of information like "Percent_Lifetime_Remain". It might be necessary to compare that actual data like "Data Units Written: 247,873,050 [126 TB]" with specs from the datasheet.

 

[rfox]

This is not much. Rotating rust allows up to 200 MB/s (or more) nowadays. NVMe can write data with a speed of a few Gigabytes per second. This really, really depends on the specific hardware - and "consumer grade" vs. "enterprise class" makes a huge difference.

Look at the SMART data: web gui - <node> - Disks - <select disk> - top area button "Show SMART values".

It should give you some hints, but not all devices give the same structure of information like "Percent_Lifetime_Remain". It might be necessary to compare that actual data like "Data Units Written: 247,873,050 [126 TB]" with specs from the datasheet.

Thanks for the quick answer! I know the nvme can handle such volumes of writes - my question was more about sustained writes at that volume (presently around 5M) constantly over a long period . . .

I checked the SMART valutes - and to my surprise , the nvme shows 13% wearout and 71TB write data ?!? Not sure if this is unusually high for 11,000 hours of operation - the nvme is a Sabrent consumer grade device:





Checking the specs of the Sabrent Rocket 4.0 ist states:

So I believe I'm good nor now!! Thx for your tips . . .

 

[cwt]

ZenArmor alone will write many files, especially when you use the local db (mongodb). If you’re running a ZFS Mirror underneath, it’s obvious that your consumer NVMEs will die sooner or later.

 

[rfox]

ZenArmor alone will write many files, especially when you use the local db (mongodb). If you’re running a ZFS Mirror underneath, it’s obvious that your consumer NVMEs will die sooner or later.

Thx! No ZFS on this box (R86s) - Small form factor and just one nvme slot - so a single 500G device using LVM & LVM-Thin

 

[UdoB]

13% wearout and 71TB write data ?!? Not sure if this is unusually high for 11,000 hours

This gives you eight more years, which (for me) is a long time. (If usage is constant over the past and the future...)

But of course you are right: decreasing the write rate will extend the remaining time.

Just as another random data point: I have Mini-PCs with PVE. Two of them write continuously 5 MB/s 24*7, while running a dozen VMs idling around. The third one writes continuously 40 MB/s because it is running a monitoring database. All of my nodes run ZFS exclusively - but with mirrored enterprise class SSDs; this one has a wearout of 7% after ~two years.

So yes, writing data for monitoring/logging purposes can be the major part of data written in a homelab...

 

[rfox]

Thanks Udo - very useful feedback!