DiskIO in CT missing

[swibwob]

Adding my voice - Disk IO show's nothing on my containers!

 

[chOzcA75vE0poCY0F6XC]

I had hoped for a nice little surprice with the release of PVE 7.1, but sadly, no change.

pve-manager/7.1-5/6fe299a0

Spoiler: pveversion -v

proxmox-ve: 7.1-1 (running kernel: 5.11.22-4-pve)
pve-manager: 7.1-5 (running version: 7.1-5/6fe299a0)
pve-kernel-5.13: 7.1-4
pve-kernel-helper: 7.1-4
pve-kernel-5.11: 7.0-10
pve-kernel-5.4: 6.4-4
pve-kernel-5.13.19-1-pve: 5.13.19-2
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-5-pve: 5.11.22-10
pve-kernel-5.11.22-4-pve: 5.11.22-9
pve-kernel-5.11.22-3-pve: 5.11.22-7
pve-kernel-5.11.22-2-pve: 5.11.22-4
pve-kernel-5.11.22-1-pve: 5.11.22-2
pve-kernel-5.4.124-1-pve: 5.4.124-1
pve-kernel-5.4.119-1-pve: 5.4.119-1
pve-kernel-5.4.114-1-pve: 5.4.114-1
pve-kernel-5.4.106-1-pve: 5.4.106-1
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: residual config
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-3
libpve-storage-perl: 7.0-15
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.14-1
proxmox-backup-file-restore: 2.0.14-1
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.4-2
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-1
pve-qemu-kvm: 6.1.0-2
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-3
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

 

[Temtaime]

Proxmox ignoring bugs ? It is there for years

 

[chOzcA75vE0poCY0F6XC]

Dominic hasn't been online on this forum since Aug 11, 2021, maybe he took some very long vacation and forgot to mention this issue to other staff members?

 

[justkru]

same here. version 7.1-4 - still not fixed.
LXCs using ZFS do not report io stats. any hotfix or manual fix possible?

 

[Prochor]

The problem is not solved, I read the information until I found a solution.

 

[Sturdily6233]

Just posting to confirm that this is still an issue.

 

[Skyrider]

Can confirm. Just a shame from the looks of it, this is being ignored.

 

[Alibek]

People:
1) diskIO stat is present in container if it is unprivileged.
2) for privileged container diskIO stat is not present.

Spoiler: pveversion

# pveversion
pve-manager/5.4-15/d0ec33c6 (running kernel: 4.15.18-30-pve)

# pveversion
pve-manager/7.2-5/12f1e639 (running kernel: 5.15.35-3-pve)

 

[Corwin]

Unfortunately diskIO stat is NOT present in container even if it is unprivileged, just checked on 7.2 (zfs system selected during installation).
Any ideas ?

 

[erfus]

Same issue here unfortunately. Running latest PVE7 with zfs. Disk Stats on VMs are fine but no Disk Stats on CT. Any way to solve this? Thanks!

 

[fabian]

see the bug report linked on the first page - there is no accounting for non-block-devices, so storages which use bind-mount(-like) semantics like ZFS don't have any usage numbers available.

 

[JEREMY_B]

https://github.com/asomers/ztop

Will this help speed up a fix?

Maybe others will find it useful until then.

 

[Ramalama]

Hey @fabian i just camed across this issue either, after like 2 years of using proxmox on all my servers xD

Wouldn't it be better to simply not display the IO Graph? Or Exchange the Graph with something like "IO is not Available with CT's on ZFS"
Everything should be better as an empty graph that looks like a bug xD

Cheers

 

[fabian]

the graph says there is no data already, and in that part we can't really differentiate between the reasons why that might be.

 

[Alibek]

May be next python script will helps proxmox developers: - https://github.com/jimsalterjrs/ioztat - it just parse /proc/spl/kstat/zfs/<pool>/objset-* (on zfs 0.8):

# ioztat -xS -s operations
                          operations    throughput      opsize   
dataset                   read  write   read  write   read  write
-----------------------  -----  -----  -----  -----  -----  -----
pool1                      578  1.43K  1.69M  39.4M  2.98K  28.1K
pool1/subvol-127-disk-0     31    541   969K  1.07M  30.8K  2.03K
pool1/subvol-453-disk-0    344     86   264K  10.2M    782   121K
pool1/subvol-453-disk-1      3    285  64.7K  48.9K  19.9K    175
pool1/subvol-604-disk-0     20    157   149K  24.9K  7.09K    162
pool1/subvol-584-disk-0     88     81  68.4K  9.68M    787   121K
pool1/subvol-327-disk-0     57     57  38.2K  7.14M    678   127K
pool1/subvol-289-disk-0      0     88    325  11.1M  1.57K   128K
pool1/subvol-584-disk-1      0     59    854  5.65K  19.9K     97
pool1/subvol-595-disk-0     10     46  17.5K  13.0K  1.59K    283
pool1/subvol-392-disk-0     20     25   156K  26.7K  7.54K  1.03K
pool1/vm-3204-disk-1         0      1      0  7.06K  17.4K     4K
pool1/vm-3204-disk-0         0      0      0  3.76K  20.1K     4K


# ioztat -xSHe -s operations
pool1    579    1433    1767710    41261671    3055    28784
pool1/subvol-127-disk-0    31    542    991777    1125362    31527    2077
pool1/subvol-453-disk-0    345    87    269856    10716516    783    123840
pool1/subvol-453-disk-1    3    285    66295    50046    20337    176
pool1/subvol-604-disk-0    21    157    152245    25480    7261    162
pool1/subvol-584-disk-0    89    82    70000    10151276    787    123809
pool1/subvol-327-disk-0    58    57    39096    7488730    679    130349
pool1/subvol-289-disk-0    0    89    325    11646770    1613    130973
pool1/subvol-584-disk-1    0    59    855    5780    20362    98
pool1/subvol-595-disk-0    11    47    17897    13278    1627    284
pool1/subvol-392-disk-0    21    26    159363    27349    7726    1054
pool1/vm-3204-disk-1    0    2    0    7229    17840    4096
pool1/vm-3204-disk-0    0    1    0    3855    20554    4096