iodelay on fresh pve 8 node

[crembz]

Hi all,

I've just installed pve 8.0 beta on a zfs nvme mirror (Samsung 980) threadripper node. I have a virtual truenas scale vm running on the local zfs store with an lsi card passed through for the zfs pool. The zfs pool is shared back to pve via nfs for isos and container templates.

I'm seeing a static io delay being reported by pve despite almost no CPU activity. What could be the cause of this:

 

[ness1602]

Your ssds are not enterprise so this is expected.

 

[crembz]

Your ssds are not enterprise so this is expected.

Sure but there is nothing running on those apart from proxmox itself. Further the iodelay seems to have no correlation to CPU usage. Also if you notice the server load, it is reporting quite high despite CPU usage being practically 0. I had a look at htop and cannot see any rogue processes or anything out of the usual.

 

[ness1602]

It is what is it, storage latency. If this is test cluster just work with it,if this is production,get better disks.
There is no magic solution for bad hardware

 

[leesteken]

Sure but there is nothing running on those apart from proxmox itself.

Proxmox is constantly logging and saving data for the graphs. Check the Syslog in the Proxmox web GUI if there are any problems being reported over and over again, like not being able to e-mail or an external storage is unreachable. Even without problems, Proxmox is quite chatty.

 

[t.lamprecht]

FWIW, we had a recent case where it was a broken autofs mount that caused a otherwise unexplained IO wait/pressure on a mostly idle host:
https://forum.proxmox.com/threads/proxmox-ve-8-0-beta-released.128677/page-5#post-563904

So, maybe check that all of your mount points are (still) valid too.

 

[crembz]

Ok so I purged the truenas iSCSI package and rebooted the server ... Iodelay is back down to 0