Yeah, a very stupid one you could say :(. Weird that I didn't notice it until now o_O . Yet another Case of the Problem being between the Chair and the Screen. Sorry about that :(.
I would have guessed that since the default none Cache Options for the Virtual Disk were used (writes ARE cached...
@fabian: spotted the Issue, almost by coincidence, while cleaning up /etc/fstab File.
There was a sync Mount Option set for the / Filesystem in /etc/fstab :rolleyes: . Not sure why I did that some Time ago ...
So essentially every single write would be forced to be written immediately to Disk...
iothread shouldn't matter and discard being disabled on the Fast VM (156 APTCacherNG) is even more weirder (it should be slower not faster if anything).
I guess it doesn't make too much of a difference at this speed and probably only relevant for TRIM Operations.
Also the Load on the Guest 151 only
If I were to take a blind Stab at it, I'd guess that only 10% of the CPU Power is actually being allocated.
Not sure how that causes the Performance of the Storage to drop THIS much though.
It's weird because from LiveUSB + chroot it worked fine, so there...
Not really.
I mean right now the entire Server load as seen from the Host is very low and most of it it's caused by running fio right now. Everything else is pretty much Idle.
The Virtual Machine 151 MirrorNAS was left unbootable because I interrupted an apt dist-upgrade and that left either the Kernel or Initramfs in a bad State (probably due to a half-compiled ZFS DKMS Module).
I basically did a rescue of that broken VM using a Virtual USB Pendrive (Debian 13...
So @fabian
First results are in for the Normal / Fast VM (156 - APTCacherNG).
Now I'm recovering the other one from a Chroot and it's MUCH faster already compared to when booted from the Virtual Disk (I booted from a Virtual USB Pendrive).
If it's an IO Issue, I'm curious as to why it only affects one VM. My initial Theory was q35 vs i440fx but that proved to not be the Case unfortunately :( .
These are old VMs and therefore still using 8k for volblocksize (both of them).
IIRC only one ZVOL is used (for MirrorNAS the other one...
Thank you @fabian for your quick Reply.
Storage Setup:
root@pve99:~# zpool status
pool: rpool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool...