/etc/pve - pmxcfs - amplification & inefficiencies

[Johannes S]

I also delight in proving myself wrong too, especially as in this case this would be of benefit to me, I own several of these https://www.kingston.com/datasheets/SEDC2000B_en.pdf and these have a TBW lifetime that is worse than these without PLP https://www.seagate.com/content/dam...firecuda-530-ssd-DS2059-5-2310-AMER-en_US.pdf

i.e whatever it is that determines nvme lifetime - it is not PLP,
I believe its whether there is a DRAM cache on the drive or not that makes the difference.

yes and without PLP data gets lost in case of a power outage. Thus drives without plp needs to write to the drive before they report a finished write. Drives with plp can give their confirmation as soon as the data is on the cache: Better performance and less writings.

At least that's my understanding from the mentioned thread. In fact the OP made a similiar Argument to yours which is reason I'm not convinced on it after rereading staff members counter arguments

 

[_gabriel]

saying its PLP that makes the difference is like saying the extra trim on my fast car are why it goes fast.

wrong, you miss the point of PLP.
it's like a RAID cache accelerator protected by battery.
PLP ensure data is always written to disk, even if crash or power loss.

 

[scyto]

wrong, you miss the point of PLP.
it's like a RAID cache accelerator protected by battery.
PLP ensure data is always written to disk, even if crash or power loss.

1. no i don't, its to protect remaining writes to be flushed from volatile cache to non-volatile storage in the event of power loss, it protects me. its sort of obvious, its in the name
2. no that's not the case there is no RAID here, it is nothing like RAID acceleration
3. yes i get that, that has nothing to do with whether it reduced the number of writes

SSDS and NVME (and spinning drives for that matter) can all be implemented with DRAM caches.
These can be implemented with or without PLP.

It is not PLP that reduced the writes, it is the DRAM cache. A capacitor doesn't store data, it has not impact beyond maintaining power to ensure writes are flushed to disk a UPS will also ensure that. PLP is designed to mitigate loss of PSU in the hardware.

 

[scyto]

Thus drives without plp needs to write to the drive before they report a finished write. Drives with plp can give their confirmation as soon as the data is on the cache: Better performance and less writings.

'less writings' only true if the data in the cache to be written is invalidated by a later write of the same block (not file, block) happens within the cache latency, which is tiny. this is not as common scenario as one would think, that's the only time a cache would save on writes, most times that would only happen to metadata in the filesystem. I assure you cache on non PLP drives also lie about the writes and also do cache invalidation if you have that enabled (and this incudes spinning HDDS too)

Do caches help performance, yes, and that's why most drives have them, and its why windows lets you control the behavior of caching too. And yes PLP for power failure write flushing is a good thing. So are UPS with scheduled shutdown. Cache and PLP might be considered synonymous by some, but it is not.

I can find no evidence that on good modern NVMEs and SSDs that there is huge difference in the write cache implementation between things like Samsung Pro NVME / SSD and drives like the Kingston with PLP.

 

[_gabriel]

no that's not the case there is no RAID here, it is nothing like RAID acceleration

RAID cache accelerator protected by battery speed up even a single disk without RAID.

it's seems you haven't tried yet a SSD PLP enterprise.
I can't write correctly in English to more explain, keep search and read, myself understand the point after many weeks if not months.

 

[waltar]

RAID cache accelerator protected by battery speed up even a single disk without RAID.

That's wrong. Which raid controller technical documentation is that stated ?

 

[_gabriel]

Cache and PLP might be considered synonymous by some, but it is not.

it's not, but there is necessarily a cache when there is PLP

 

[scyto]

it's not, but there is necessarily a cache when there is PLP

agree 100% that doesn't make PLP a feature that 'saves on disk writes'

 

[_gabriel]

That's wrong. Which raid controller technical documentation is that stated ?

I mean speed up volume(s) ONLY on writes : sync writes and small writes, thanks to DRAM cache of the RAID controller protected by battery.

 

[_gabriel]

agree 100% that doesn't make PLP a feature that 'saves on disk writes'

Not fully understand as I'm not English, but 'saves on disks writes' are done indirectly, because cache optimize data before write cells.

 

[scyto]

it's seems you haven't tried yet a SSD PLP enterprise.

thanks i have, you are asserting PLP is what gives something like the soldigm drives high PBW, low latency etc - it isn't, that would be the controller and the modules and their choice of cache strategy

*all* PLP does is a battery backup to flush what the controller and cache is already doing, that's it, nothing else

if you think that is wrong please find me an architecture article from one of the 'enterprise SSD / NVME providers' where they state explicitly it is PLP (a capacitor and a power circuit) that causes less drive wear and tear.... i will wait.

 

[scyto]

Not fully understand as I'm not English, but 'saves on disks writes' are done indirectly, because cache optimize data before write cells.

Caches don't optimize data before writes. they just keep a read cache and write cahce of recent blocks incase they are needed again, write back can reduce latency, it won't reduce disks writes unless a block in cache is thrown away

PLP protects a write cache, that's it, you can buy drives that have write cache and don't have PLP and they will boradlu work the same if it has write back enabled, PLP drives may come with write back enabled by default - thats likely the source of why you think 'PLP is magic'

 

[scyto]

hanks to DRAM cache of the RAID controller protected by battery.

I am glad you finally agree DRAM cache and Battery Backup (capacitor backup in the case of hardware PLP) are two different features.

 

[_gabriel]

we are agree.
it seems more a wording problem than a wrong assertion.

 

[scyto]

My last comment on this, this is why this issue rates a 'meh' (though i love the bug fixed that were done).

The % of writes from pmxcfs is just too low as a % of all writes to give much thought to, this is 24 hours of iotop, this will affect my drive life in no meaningful way.

 

[waltar]

The problem is not the amount of data as of the amount of small writes by pmxcfs which results in internal cell rewrites which reduces the ssd lifetime.

 

[scyto]

The problem is not the amount of data as of the amount of small writes by pmxcfs which results in internal cell rewrites which reduces the ssd lifetime.

thanks, what's the relationship of FS blocks to size / number of cells?

also PLP isn't going to change what cells get written to it, it is just a capacitor and a power circuit, the cache may if the cache discards a block. Interestingly all my PLP NVMe (kingston) here default to write through and my non-plp drives (firecuda) default to write back. Making this an even more intriguing conversation to me.

 

[waltar]

That's different from ssd to ssd and even models while dependent on used cell technologie, ssd controller and firmware.