Ok, so: There's definitively something odd with the block layer statistics for phyiscal devices in these kernel versions (7.0.14-8 now).
In my log, I see sudden large jumps in write ticks and time_in_queue values, like in this example (leading...
Munin is a basic monitoring tool that mostly consists of a daemon running plugins (scripts) that produce output in Munin format, and a data collector that polls for this information, usually every five minutes. The collector writes everything...
This creates spikes that are well above munin defaults for sending emails. And these defaults are set by munin maintainers ?
It's 0:3 by default (warning/email if above 3 seconds). It's like that since 20 years I believe...
WARNINGs: Write IO...
If this ever gets to a kernel developer : this is super easy to replicate ; just add munin package on a vm or LXC, or even on the host itself.
monitoring disk latency is default.
pve devs say for -6
update submodules and patches to...
I'm back at -5 and the problem vanished .
So basically the problem appeared in -6 (as per Galaxis) , is still there in -8, I was just a bit late in the game.
If it's a new kernel feature and not the sign of the disks getting crazy, it sure...
Since @weppa writes it was still fine in -5 (which I skipped coming from -4), and the effect first showed up for me in -6, something must have happened between those two versions.
@Galaxis you're not crazy ;
https://forum.proxmox.com/threads/increased-latency-on-disks-since-9-2-6-and-associated-7-0-x-kernel.185498/
That is my own topic I created.
It's basically unusable (I've got 440 emil alerts this morning) and very...
Thanks @Neobin !
In the meantime
proxmox-boot-tool kernel pin 7.0.14-5-pve
proxmox-boot-tool refresh
UPDATE : I can confirm the problem DISAPPEARS with kernel 7.0.14-5
So it's specific to kernels after this one.
I have a couple of proxmox VE hosts, running 9.2.4 with 7.0 kernel.
These systems are updated regularly (like once a month) with proper reboot
For the last update (9.2.4 to 9.2.6 - on the 7.0 kernel) I have something weird happening that trigger...