Unexpected disk util 100%

[Andrii]

Have not high iops usage, the disk latency was fine, but disk util 100%. I saw that two times with different nodes and only on LXC. Why it can be? Is that a bug of LXC?

 

[dcsapak]

mhmm i do not really get the units of your utilization graph

it says "% busy" on the left hand side, but below it says "470000 max"

what is this 470000? how does it calculate the % and where does the data come from?

 

[Andrii]

Sorry, I said about %busy of course.
It is a bug of munin. When you use all resources you have it can write incorrect information. Must be 100% of course.
Can't access to the server at this moment.

 

[Andrii]

The same issue on the other server one day ago.

 

[Andrii]

During all time, when the disk busy was 100%, was many kernel errors.

Short video - https://screencast.com/t/a6iELIP9l9dl

 

[dcsapak]

do you have a backup job running at this time? or anything else in the logs?

 

[Andrii]

do you have a backup job running at this time? or anything else in the logs?

No.

 

[Andrii]

I could not find the reason why this happened. LXC is a very unstable. We migrated 5 nodes from OpenVZ to LXC and now have a downtime every 2 days.

 

[dcsapak]

how do the container configs look like?

 

[Andrii]

#Start
arch: amd64
cmode: console
console: 1
cores: 1
cpulimit: 0
cpuunits: 1024
hostname: server
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,gw=185.162....1,hwaddr=9E:7E:27:93:3A:F8,ip=185.162...../24,rate=6,type=veth
onboot: 1
ostype: centos
rootfs: lvmthin:vm-500-disk-1,size=10G
swap: 0