VM shows 0.0% disk usage

Amadex · Sep 8, 2022

Hello, my VM shows 0.0% disk usage and it's a 50GB VM in production (sites, emails, etc.). Qemu Guest Agent is installed. What could it be wrong? It's on a rented dedicated server.

datschlatscher · Sep 8, 2022

At the moment, disk usage for VMs is not tracked and therefore this value is always displayed as 0%.

There is still an active issue on this problem. Please see this issue on our bug tracker for more information or if you want to contribute.

Amadex · Sep 8, 2022

datschlatscher said:
At the moment, disk usage for VMs is not tracked and therefore this value is always displayed as 0%.

There is still an active issue on this problem. Please see this issue on our bug tracker for more information or if you want to contribute.

Thanks for the answer. I've switched from Microsoft Hyper-V to Proxmox, so questions questions.

Deepen Dhulla · Sep 8, 2023

Work-Around Fix for Ceph-Storage Usage for VM Usage in Proxmox Virtual Environment (PVE) Cluster.
Would submit the code in bugzilla too (will have to check Development guideline too)
and would be working on ZFS storage too, soon ; as its another popular in PVE Cluster setup,
sharing it if any one want to refer and want to use and want to give feedback.

Introduction:

This document provides step-by-step instructions on implementing a
work-around fix for Ceph-storage utilization in a Proxmox Virtual Environment (PVE) cluster.
Specifically, this fix addresses the issue of displaying boot disk usage for virtual machines (VMs),
including snapshot space, in a Ceph storage configuration.

Procedure:

1. Backup the Original Perl File:
Before making any changes, it is crucial to create a backup of the original Perl file for safety purposes.

Code:

cp /usr/share/perl5/PVE/QemuServer.pm /opt/QemuServer-original-`date +%Y-%m-%d-%s`.pm

2. Edit the Perl File:
Open the Perl file for editing. You can use any text editor, but we will use the example of using the vi editor.

Code:

vi /usr/share/perl5/PVE/QemuServer.pm

Inside the editor, locate the line containing "{disk}" (approximately around line 2944).

3. Modify the Perl Code:

You will see the following code block:

Code:

my $size = PVE::QemuServer::Drive::bootdisk_size($storecfg, $conf);
if (defined($size)) {
    $d->{disk} = 0; # no info available
    $d->{maxdisk} = $size;
} else {
    $d->{disk} = 0;
    $d->{maxdisk} = 0;
}

After the comment line "# no info available," add the following code to retrieve disk usage from the Ceph pool for VMs:

Code:

##### CODE TO FETCH VM DISK USAGE FROM CEPH POOL START #####
my @bootdiskorder = split('=', $conf->{boot});
my @bootdiskname = split(';', $bootdiskorder[1]);
my @bootdiskinfo = split(",", $conf->{$bootdiskname[0]});
my @bootdiskdetail = split(":", $bootdiskinfo[0]);
my $bootdiskstorage = $bootdiskdetail[0];
my $bootdiskimage = $bootdiskdetail[1];

if (defined $storecfg->{ids}->{$bootdiskstorage}->{type}) {
    my $bootdisktype = $storecfg->{ids}->{$bootdiskstorage}->{type};
    my $bootdiskpool = $storecfg->{ids}->{$bootdiskstorage}->{pool};

    if ($bootdisktype eq "rbd") {
        my $cephrbddiskinfocmd = "rbd disk-usage -p " . $bootdiskpool . " " . $bootdiskimage . " --format=json";
        my $cephrbddiskinfo = `$cephrbddiskinfocmd`;
        $cephrbddiskinfo =~ s/\n/""/eg;
        $cephrbddiskinfo =~ s/\r/""/eg;
        $cephrbddiskinfo =~ s/\t/""/eg;
        $cephrbddiskinfo =~ s/\0/""/eg;
        $cephrbddiskinfo =~ s/^[a-zA-z0-9,]//g;
        my $total_used_size = 0;

        if ($cephrbddiskinfo =~ /$bootdiskimage/) {
            my $cephrbddiskinfoarray = decode_json($cephrbddiskinfo);
            foreach my $image (@{$cephrbddiskinfoarray->{'images'}}) {
                if (defined $image->{'used_size'}) {
                    $total_used_size += $image->{'used_size'};
                }
            }
            $d->{disk} = $total_used_size;
        }
    }
}
##### CODE TO FETCH VM DISK USAGE FROM CEPH POOL END #####

4. Restart the pvestatd Service:

After making the necessary changes, restart the pvestatd service to apply the modifications.

Code:

systemctl restart pvestatd.service

5. Check for Errors:

Monitor the system logs for any potential errors to ensure that the changes were applied without issues.

Code:

tail -f /var/log/syslog

6. Verify Disk Usage:

If everything is functioning correctly, you should now be able to see the disk usage, including boot disk usage and percentage used, for VMs when they are in the "ON" state.

I hope it help some, and we get feedback too.
Thanks in advance.
-Deepen.

mithrandir_63 · Feb 7, 2024

Hello everybody,

Thank you very much for the post and the solution.
But after seriously following it step by step, it didn't work for me.

I'm digging up this post! !
here are 3 screenshots:
- the code inserted in the "QemuServer.pm" file

- the table with all the vms where we can clearly see that disk usage is anamorally at 0%.
we see that on the other hand it still works correctly for the “LXC” container :

- the results of "tail -f /var/log/syslog" which shows the absence of log files and therefore no bug! ! !

Thank you in advance for your help.

Sincerely
@+++

Deepen Dhulla · Feb 8, 2024

I hope your storage uses Ceph RBD, as the code is currently designed exclusively for Ceph Storage (I will publish ZFS Storage compatibility soon), authored by me. Additionally, for syslog, which is not installed by default in PVE8, you will need to install the rsyslog package.

jmcruvellier · Jun 6, 2024

Deepen Dhulla said:
I hope your storage uses Ceph RBD, as the code is currently designed exclusively for Ceph Storage (I will publish ZFS Storage compatibility soon), authored by me. Additionally, for syslog, which is not installed by default in PVE8, you will need to install the rsyslog package.

Hi, have you been able to publish the compatibility code somewhere? I've searched for it but without success.
Thanks for you great work!

Deepen Dhulla · Jun 11, 2024

Complete code with ceph rbd and ZFS is as below , keep in mind zfs would show more size than actual allocated , example 100GB is take as 155GB , is snapshot is attached.

Code:

##### CODE TO FETCH VM DISK USAGE FROM CEPH + ZFS POOL START #####
my @bootdiskorder = split('=', $conf->{boot});
my @bootdiskname = split(';', $bootdiskorder[1]);
my @bootdiskinfo = split(",", $conf->{$bootdiskname[0]});
my @bootdiskdetail = split(":", $bootdiskinfo[0]);
my $bootdiskstorage = $bootdiskdetail[0];
my $bootdiskimage = $bootdiskdetail[1];

if (defined $storecfg->{ids}->{$bootdiskstorage}->{type}) {
    my $bootdisktype = $storecfg->{ids}->{$bootdiskstorage}->{type};
    my $bootdiskpool = $storecfg->{ids}->{$bootdiskstorage}->{pool};
    if ($bootdisktype eq "zfspool") {
        my $zfsdiskinfocmd ="zfs get -H -p -oname,value  used ".$bootdiskpool."/".$bootdiskimage;
        my $zfsdiskinfo=`$zfsdiskinfocmd`;
        $zfsdiskinfo =~ s/\n/""/eg;
        $zfsdiskinfo =~ s/\r/""/eg;
        my $total_used_size = 0;
        if ($zfsdiskinfo =~ /$bootdiskimage/) {
                my @zfsdiskbytes=split("\t",$zfsdiskinfo);
                $total_used_size=$zfsdiskbytes[1];
                }
        $d->{disk} = $total_used_size;
        }
   if ($bootdisktype eq "rbd") {
        my $cephrbddiskinfocmd = "rbd disk-usage -p " . $bootdiskpool . " " . $bootdiskimage . " --format=json";
        my $cephrbddiskinfo = `$cephrbddiskinfocmd`;
        $cephrbddiskinfo =~ s/\n/""/eg;
        $cephrbddiskinfo =~ s/\r/""/eg;
        $cephrbddiskinfo =~ s/\t/""/eg;
        $cephrbddiskinfo =~ s/\0/""/eg;
        $cephrbddiskinfo =~ s/^[a-zA-z0-9,]//g;
        my $total_used_size = 0;
        if ($cephrbddiskinfo =~ /$bootdiskimage/) {
            my $cephrbddiskinfoarray = decode_json($cephrbddiskinfo);
            foreach my $image (@{$cephrbddiskinfoarray->{'images'}}) {
                if (defined $image->{'used_size'}) {
                    $total_used_size += $image->{'used_size'};
                }
            }
            $d->{disk} = $total_used_size;
        }
    }
}
##### CODE TO FETCH VM DISK USAGE FROM CEPH POOL END #####

savalet_ · Jul 25, 2024

You can use my modified /usr/share/perl5/PVE/QemuServer.pm file to use the qemu guest agent to retrieve the disk usage:
https://gist.github.com/savalet/f1f34a9de9dfb331505294d96bae9f33

Deepen Dhulla · Jul 26, 2024

Nice point get it from Quest-Agent, can you share section change.. so respective code can be taken ..also does this shows all extra attached disk or only boot disk ?

vizerei · Nov 5, 2024

So looking through this and other threads, and at the bug report it seems this issue has been going on for 5 years. For such a basic hypervisor functionality is there any way to get this looked at with more priority? Not to be obtuse, but we're paying full licensing for 7 servers and this being outstanding and a weird workaround being the only fix is disappointing.

LnxBil · Nov 5, 2024

vizerei said:
So looking through this and other threads, and at the bug report it seems this issue has been going on for 5 years. For such a basic hypervisor functionality is there any way to get this looked at with more priority? Not to be obtuse, but we're paying full licensing for 7 servers and this being outstanding and a weird workaround being the only fix is disappointing.

Maybe the clue that it's not implemented already should tip you off that it is not as easy as it seems to. Using the disk usage from qemu is just wrong, as pointed out here. It's the same with the memory usage.

vizerei · Nov 6, 2024

LnxBil said:
Maybe the clue that it's not implemented already should tip you off that it is not as easy as it seems to. Using the disk usage from qemu is just wrong, as pointed out here. It's the same with the memory usage.

If there is a recommended workaround that customers are expected to generally perform then I don't think that "it's not as easy as it seems" holds much water. That could be automated behind the scenes.

LnxBil · Nov 6, 2024

vizerei said:
If there is a recommended workaround that customers are expected to generally perform then I don't think that "it's not as easy as it seems" holds much water. That could be automated behind the scenes.

There is no workaround, that's the problem. For e.g. thick-LVM, there will always be 100% disk usage. I don't think that "information" will help anyone.

akballow · Dec 3, 2024

Deepen Dhulla said:

Complete code with ceph rbd and ZFS is as below , keep in mind zfs would show more size than actual allocated , example 100GB is take as 155GB , is snapshot is attached.

Code:

##### CODE TO FETCH VM DISK USAGE FROM CEPH + ZFS POOL START #####
my @bootdiskorder = split('=', $conf->{boot});
my @bootdiskname = split(';', $bootdiskorder[1]);
my @bootdiskinfo = split(",", $conf->{$bootdiskname[0]});
my @bootdiskdetail = split(":", $bootdiskinfo[0]);
my $bootdiskstorage = $bootdiskdetail[0];
my $bootdiskimage = $bootdiskdetail[1];

if (defined $storecfg->{ids}->{$bootdiskstorage}->{type}) {
    my $bootdisktype = $storecfg->{ids}->{$bootdiskstorage}->{type};
    my $bootdiskpool = $storecfg->{ids}->{$bootdiskstorage}->{pool};
    if ($bootdisktype eq "zfspool") {
        my $zfsdiskinfocmd ="zfs get -H -p -oname,value  used ".$bootdiskpool."/".$bootdiskimage;
        my $zfsdiskinfo=`$zfsdiskinfocmd`;
        $zfsdiskinfo =~ s/\n/""/eg;
        $zfsdiskinfo =~ s/\r/""/eg;
        my $total_used_size = 0;
        if ($zfsdiskinfo =~ /$bootdiskimage/) {
                my @zfsdiskbytes=split("\t",$zfsdiskinfo);
                $total_used_size=$zfsdiskbytes[1];
                }
        $d->{disk} = $total_used_size;
        }
   if ($bootdisktype eq "rbd") {
        my $cephrbddiskinfocmd = "rbd disk-usage -p " . $bootdiskpool . " " . $bootdiskimage . " --format=json";
        my $cephrbddiskinfo = `$cephrbddiskinfocmd`;
        $cephrbddiskinfo =~ s/\n/""/eg;
        $cephrbddiskinfo =~ s/\r/""/eg;
        $cephrbddiskinfo =~ s/\t/""/eg;
        $cephrbddiskinfo =~ s/\0/""/eg;
        $cephrbddiskinfo =~ s/^[a-zA-z0-9,]//g;
        my $total_used_size = 0;
        if ($cephrbddiskinfo =~ /$bootdiskimage/) {
            my $cephrbddiskinfoarray = decode_json($cephrbddiskinfo);
            foreach my $image (@{$cephrbddiskinfoarray->{'images'}}) {
                if (defined $image->{'used_size'}) {
                    $total_used_size += $image->{'used_size'};
                }
            }
            $d->{disk} = $total_used_size;
        }
    }
}
##### CODE TO FETCH VM DISK USAGE FROM CEPH POOL END #####

I am not smart enough to understand what is going on yet as I am learning but I was unable to get the storage info for my redhat 9 vms specifically. all others seems to be fine. They all have the qemu-agent. Any way to troubleshoot this?

For example for the line

Code:

my $zfsdiskinfocmd ="zfs get -H -p -oname,value  used ".$bootdiskpool."/".$bootdiskimage;

All the VMs including the RedHat ones show used space value, but it is defaulting to 0 because maybe the earlier function

Code:

my $size = PVE::QemuServer::Drive::bootdisk_size($storecfg, $conf);
if (defined($size)) {
    $d->{disk} = 0; # no info available
    $d->{maxdisk} = $size;
} else {
    $d->{disk} = 0;
    $d->{maxdisk} = 0;
}

Unless somehow the storage config part from here

Code:

if (defined $storecfg->{ids}->{$bootdiskstorage}->{type}) {

is not being read correctly as well. I am just playing with things for now but hope someone can offer some help

EDIT:

ISSUE WAS BOOT ORDER
so cdrom was first boot item, which it should be the case normally right? But i guess changing it to be after disk or disabled will fix this.

Search

Search

VM shows 0.0% disk usage

Amadex

New Member

datschlatscher

Proxmox Retired Staff

Amadex

New Member

Deepen Dhulla

Renowned Member

mithrandir_63

New Member

Deepen Dhulla

Renowned Member

jmcruvellier

New Member

Deepen Dhulla

Renowned Member

savalet_

New Member

Deepen Dhulla

Renowned Member

vizerei

Renowned Member

LnxBil

Distinguished Member

vizerei

Renowned Member

LnxBil

Distinguished Member

akballow

New Member

We value your privacy