Reduce writes on main drive(SSD), how?

[b0n3v]

Hello to all, first thank to all of you for this excellent product!

I have clean install of Proxmox VE 6.0 without any VM and CT, with 2 ZFS mirror pools.
I run "iotop -Pao" and i see something strange to me, two procees write on my SSD every 3-6 seconds.
kworker and jbd2/dm-1-8 . Yes the writes isn't much in size but is frequency, and my SSD will wear to fast.
I already disabled this two procces: "pve-ha-crm" and "pve-ha-lrm", made cron for trim task every week, and reduced swap to 10%.
I want to optimize everything about writing on the SSD. I am opend for any opinions and guide.
Im looking for production state for my home lab.

I already see this link - LINK - but i don't know it's applicable?

My specs:
Ryzen 1700
32GB DDR4 noECC 3266mhz
2x1TB Sata HDD (ZFS Mirror lz4)
2x2TB Sata HDD (ZFS MIrror lz4)
M2 nvme SSD 512GB

Thanks to all!

 

[tim]

kworker..kernel worker threads.
jbd2.. journaling

My only advice in this context is, if you don't use a SD Card as your system disk this is probably a waste of time, please don't get me wrong not meant offensive in any way, but your M2 nvme will probably last years with or without "optimizations" in your home lab.

 

[SamirD]

kworker..kernel worker threads.
jbd2.. journaling

My only advice in this context is, if you don't use a SD Card as your system disk this is probably a waste of time, please don't get me wrong not meant offensive in any way, but your M2 nvme will probably last years with or without "optimizations" in your home lab.

I second this as modern (and even some not so modern) ssds will last years even with writes. And with the prices on these dropping all the time, you're spending a lot of time to save what eventually will be a $20 piece of hardware--not worth the time.

 

[guletz]

Hi,

Yes if 20$ is not a problem, and you do not want to use better your money, is ok (I would prefer to donate this money for a better cause, but anybody can decide what to do with their own money).

For any other reason you could do something like this:

I guess you have zfs instaleed using the 2 hhd:
- backup your /var/log to /var/log.bkp (to be safe)
- stop the syslog service
- make a new dataset in rpool like rpool/syslog
- delete /var/log
- create a symlink for /var/log -> /rpool/syslog

The same ideea for /tmp, and for rrd database and for pmxfs.


Good luck / Bafta

 

[SamirD]

$20 is an arbitrary number. At the rate ssds are dropping in price a 512GB nvme ssd will be $20 by Christmas, and worth nearly nothing by the time it wears out--and that was my point. There will also be version updates that may resolve extra writes, or increase them, or make them mandatory by the time you make the changes.

Hardware increasingly becomes a commodity over time. I remind myself of that fact when I look at my $50 HP DL380 G5 that was originally over $7500 brand new.

 

[sumguy]

For systems without SSDs, do the constant writes to spinning disk impact performance much? Anyone test this out? Increase IO delay?

They still don't make any flash drives that are reliable enough to run OSes like proxmos on?

 

[UdoB]

They still don't make any flash drives that are reliable enough to run OSes like proxmos on?

For sure they do!

All (most) "Enterprise class" / "Datacenter Edition" labeled SSDs will do. The point is: bought new they may cost a fortune - several thousands $/€ for a few TB is "normal".

At home I am running used(!) Intel SSDs (for example: the relative old DC S3610 series) with ZFS and haven't had a single problem yet. My paranoia-level is high, so I do mirror them nevertheless...

 

[sumguy]

Hey UdoB thank you for your reply. I am aware of those types of drives - I own quite a few of them myself. I was speaking of USB flash sticks when I said flash drives. Sorry for not being clearer. There are some cases where space/cost wise a usb flash stick would be way better suited than an SSD, and some cases where SSD is not even an option but a USB stick is.

 

[UdoB]

I was speaking of USB flash sticks

Ah! Okay.
Classic USB sticks are never a recommendation for more than one reason. If you need to use them you may just mirror (or triple-mirror!) them. Try to use more than one USB controller - if the documentation tells you which port is connected to which controller chip. Or connect one stick at the front and others at the rear. Use different manufacturers.

I have never used normal USB sticks in this PVE context. But I am using two external USB-NVMe adapters with USB3 as ZFS mirror with one of my "servers" @home - because of lack of internal SATA or M.2 connectors. (As cache for rotating rust...)

 

[wolfspyre]

ohai!

I take something of a different approach than most, in that I try to push a bunch of my systemic logs off the host and not DURABLY log locally.

This is especially useful for embedded hosts/rpi's with usb/sd/microsd storage for localdisk.. but also useful in cases like this... (imo)
the azlux repo of awesomeness(tm)
http://packages.azlux.fr/
contains some astoundingly cool tools (check it out)..

the one I might encourage exploring is: https://github.com/azlux/log2ram

basically just creates a ramdisk for the storage of /var/log.. so you eat a lil memory but logwriting's a lot faster.... I don't **REALLY** care about persisting them, as I'm pushing them to a loghost for durable storage.

this works for me, but it isn't a strategy for everyone....