[SOLVED] What causes these gaps?

[edtrumbull]

If I take a look at my cluster - seven nodes, three of which are ceph storage, the other four house vms only, I see gaps in the summary graphs when I would not expect to - the gaps show up when I haven't shut down any nodes, and if I check uptimes on the nodes they have not rebooted.

I'm attaching a screenshot of the behavior.
I will be happy to give any information that may help figure this out.

 

[gfngfn256]

Your summary graphs are from a VM (100) on a node (kvm-03)
To show summary graphs from a node, click on the node and then on "Summary".

 

[edtrumbull]

Correct. The gaps show up in the VM graphs. They also show up in the node graphs, but I didn't capture that screenshot.
The question remains - what's causing gaps in the graphs when the object being represented has not been moved or shut down during the period surveyed?

 

[gfngfn256]

Are these nodes/VMs accessible directly (not through cluster) during these breaks?

 

[spirit]

the pvestatd daemon is doing the stats.

maybe look at the service

journalctl -u pvestatd


Also, sometime, a buggy/hanging storage can slowdown or hand pvestatd (you should have holes in all graphs, vms or host in this case)

 

[edtrumbull]

Yep.
They can be accessed. They don't appear to be unhappy in the least.

 

[edtrumbull]

I suspect that this was related to this thread: https://forum.proxmox.com/threads/pve-7-4-17-gui-too-many-redirections-error.140695/

The solution was to establish a separate network for corosync. When that was completed the gaps stopped.