Constant Disk I/O (wondering if it's okay)

[smonroe]

Hello, I've just started getting into the world of self hosting and so my understanding of this field is basic at best.

So here's my story leading up to my concern.
I let my server run 24/7, I've had it up for maybe 3 months now. Last night I was trying to sleep and I noticed, my server was super loud. Much louder than I've ever noticed before and it was constant. Me not knowing much about hardware I wasn't sure what component was making the noise. It was a rattling sound that would last only a moment but would occur repeatedly every moment or two, so it was a rapid on-off-on-off-on succession of rattlings. I opened up the case and listened around. I put my fingers on the HDD and noticed that it vibrated at the exact moments the rattling noise occurred so I reckoned it was the cause. Then I noticed a light on top of the case turned red during the rattlings/HDD vibrations as well. I believe that light represents I/O for the disk. So at this point I was pretty certain the constant noise was constant disk I/O.

So next I wanted to determine what was causing the constant I/O. I installed iotop after reading recommendations for it for I/O monitoring. After running iotop -a I saw [txg_sync] to be the top process but it was only taking 3-10% I/O. And nothing else was even taking so much as a single percent. I wouldn't expect that low of I/O to result in constant rattling and for the HDD I/O light to be activating rapidly, non-stop. And this goes on for hours, again, non-stop, not in spurts - incessantly.

I've read multiple places that I/O during idle times is normal; system jobs doing their thing in the background. But is incessant I/O normal? If it is I'll just learn to tune out the sound and rest in peace knowing my server is fine.

But one other bit of potentially important information which I found concerning is, I discovered that proxmox says my SSD has 100% wearout. I don't know what that means. Sounds very bad. However my server is working fine from what I can tell, other than the unabating rattling which as I explained I've deduced to be I/O. Please let me know any thoughts on the matter and how I can understand what is going on better to get some peace of mind on the matter and fix the problem if there is one.

Thank you for considering my story and helping me understand what is going on, very much appreciated!

 

[William Edwards]

But one other bit of potentially important information which I found concerning is, I discovered that proxmox says my SSD has 100% wearout. I don't know what that means. Sounds very bad.

It depends on the S.M.A.R.T. value provided by your disk vendor. If that value has always been 100%, there's probably no need to worry.

 

[H4R0]

smart wearout starts at 100% and goes down to 0%

proxmox writes almost permanently to disk due to the gui graphs.

 

[smonroe]

smart wearout starts at 100% and goes down to 0%

proxmox writes almost permanently to disk due to the gui graphs.

From what I could see it seemed like txg sync was the culprit and I read about it a bit although didn't see much info on it but looks like it is something to do with my zfs file storage syncing things? Any way for it to do that less? Or if it's proxmox graphs, any way I could make that less frequent or even off? I don't really care about the graphs

 

[H4R0]

From what I could see it seemed like txg sync was the culprit and I read about it a bit although didn't see much info on it but looks like it is something to do with my zfs file storage syncing things? Any way for it to do that less? Or if it's proxmox graphs, any way I could make that less frequent or even off? I don't really care about the graphs

txg sync is zfs doing its writes to the disk.

You could try disabling pvestatd, dunno if that disables the rrd graphs completly.

Another method is to increase sync timeout of zfs "zfs_txg_timeout", but that could lead to data loss on power failure.

 

[smonroe]

You could try disabling pvestatd, dunno if that disables the rrd graphs completly.

Another method is to increase sync timeout of zfs "zfs_txg_timeout", but that could lead to data loss on power failure.

Hmm, yeah seems like it's all behaving as expected and better left alone at least by me not really knowing what I'm doing. I've already started zoning out the rattling sounds at this point. It's generally only at night when I'm trying to sleep that I notice it. Going to consider moving it into another room. Thanks for the deeper dive info on what's going on here.

 

[H4R0]

Hmm, yeah seems like it's all behaving as expected and better left alone at least by me not really knowing what I'm doing. I've already started zoning out the rattling sounds at this point. It's generally only at night when I'm trying to sleep that I notice it. Going to consider moving it into another room. Thanks for the deeper dive info on what's going on here.

What kind of setup do you got in your sleeping room ? I doubt its a rack server ?

You are probably better of installing proxmox root on a ssd, less wear and no noise.

If it is a consumer hdd it could die rather fast, make sure to use raid.

Better use enterprise sas drives or something like wd purple which are optimized for constant writes, if you really want to go with a hdd.