ZFS Storage Summary for better understanding.

[Deepen Dhulla]

I am sharing a quick summary and the corresponding PHP code that I found helpful.
After installing php-cli, this script provided clear insight into my current thin provisioning status.
I hope this proves useful for anyone managing similar storage configurations.

=================================================================
ACTUAL 'VMSTORAGE' STORAGE CAPACITY SUMMARY (VMs + CTs)
=================================================================
Total Pool/Dataset Capacity (100%) : 5.69 TB
Total Space Currently Free : 3.15 TB
Total Space Used (Pool Level) : 2.54 TB (44.67% of 100%)
-----------------------------------------------------------------
TOTAL Provisioned / Allocated : 6.43 TB (113.07% of capacity [OVERCOMMITTED])
- VM Disks (volsize) : 4.84 TB
- CT Containers (refquota) : 1.59 TB
-----------------------------------------------------------------
TOTAL Actual Space Used (VMs + CTs) : 2.54 TB (44.66% of 100%)
- Active Data (Referenced) : 2.54 TB
- VM Disk Consumption : 1.77 TB
- CT Container Consumption : 790.39 GB
=================================================================


Below PHP Code :

<?php

$target = isset($argv[1]) ? $argv[1] : 'vmstorage';

/**
 * Format bytes to human-readable format.
 */
function formatBytes($bytes, $precision = 2)
{
    $units = array('B', 'KB', 'MB', 'GB', 'TB', 'PB');
    $pow = $bytes > 0 ? (int)floor(log($bytes, 1024)) : 0;
    $pow = min($pow, count($units) - 1);
    $bytes /= pow(1024, $pow);
    return round($bytes, $precision) . ' ' . $units[$pow];
}

/**
 * Execute command and return clean output lines.
 */
function runCmd($cmd)
{
    $output = array();
    $returnVar = 0;
    exec($cmd . ' 2>&1', $output, $returnVar);

    if ($returnVar !== 0) {
        fwrite(STDERR, "Error executing: {$cmd}\n" . implode("\n", $output) . "\n");
        exit(1);
    }

    return array_values(array_filter(array_map('trim', $output)));
}

// 1. Storage Pool Capacity
$targetInfo = runCmd("zfs list -Hp -o used,avail " . escapeshellarg($target));
if (empty($targetInfo)) {
    fwrite(STDERR, "Could not retrieve capacity info for '{$target}'.\n");
    exit(1);
}

list($storageUsed, $storageAvail) = explode("\t", $targetInfo[0]);
$totalCapacityBytes = (float)$storageUsed + (float)$storageAvail;

// 2. Query both filesystems (CTs) and volumes (VMs)
// Properties: type, name, volsize, refquota, quota, referenced, usedbysnapshots, used
$cmd = sprintf(
    'zfs list -Hp -r -t filesystem,volume -o type,name,volsize,refquota,quota,referenced,usedbysnapshots,used %s',
    escapeshellarg($target)
);
$lines = runCmd($cmd);

$vmAllocated = 0.0;
$vmUsed      = 0.0;
$ctAllocated = 0.0;
$ctUsed      = 0.0;

$items = array();

foreach ($lines as $line) {
    $cols = explode("\t", $line);
    if (count($cols) < 8) {
        continue;
    }

    list($type, $name, $volsize, $refquota, $quota, $referenced, $usedbysnapshots, $used) = $cols;

    // Skip the root target dataset itself
    if ($name === $target) {
        continue;
    }

    $referencedBytes = (float)$referenced;
    $snapBytes       = (float)$usedbysnapshots;
    $usedBytes       = (float)$used;
    $baseName        = basename($name);

    if ($type === 'volume') {
        // VM Disk
        $allocBytes = (float)$volsize;
        $vmAllocated += $allocBytes;
        $vmUsed      += $usedBytes;
        $kind        = 'VM (zvol)';
    } else {
        // Container (LXC subvolume) or Dataset
        // LXCs use refquota or quota for provisioned size. If neither is set, fall back to referenced.
        $qBytes = (float)$refquota > 0 ? (float)$refquota : (float)$quota;
        $allocBytes = $qBytes > 0 ? $qBytes : $referencedBytes;
        $ctAllocated += $allocBytes;
        $ctUsed      += $usedBytes;
        $kind        = (strpos($baseName, 'subvol-') === 0) ? 'CT (subvol)' : 'Dataset';
    }

    $items[] = array(
        'name'       => $baseName,
        'kind'       => $kind,
        'allocated'  => $allocBytes,
        'referenced' => $referencedBytes,
        'snapshots'  => $snapBytes,
        'used'       => $usedBytes,
    );
}

// 3. Print Detailed List
printf("\n%-32s | %-12s | %-14s | %-12s | %-12s\n", "Name", "Type", "Allocated", "Referenced", "Total Used");
echo str_repeat('-', 94) . "\n";
foreach ($items as $item) {
    printf(
        "%-32s | %-12s | %-14s | %-12s | %-12s\n",
        substr($item['name'], 0, 32),
        $item['kind'],
        formatBytes($item['allocated']),
        formatBytes($item['referenced']),
        formatBytes($item['used'])
    );
}

// 4. Summaries & Percentages
$combinedAllocated = $vmAllocated + $ctAllocated;
$combinedUsed      = $vmUsed + $ctUsed;

$poolUsedPct = $totalCapacityBytes > 0 ? ((float)$storageUsed / $totalCapacityBytes) * 100 : 0;
$combAllocPct = $totalCapacityBytes > 0 ? ($combinedAllocated / $totalCapacityBytes) * 100 : 0;
$combUsedPct  = $totalCapacityBytes > 0 ? ($combinedUsed / $totalCapacityBytes) * 100 : 0;

echo "\n" . str_repeat('=', 65) . "\n";
printf(" ACTUAL '%s' STORAGE CAPACITY SUMMARY (VMs + CTs)\n", strtoupper($target));
echo str_repeat('=', 65) . "\n";
printf("Total Pool/Dataset Capacity (100%%)   : %s\n", formatBytes($totalCapacityBytes));
printf("Total Space Currently Free           : %s\n", formatBytes((float)$storageAvail));
printf("Total Space Used (Pool Level)        : %s (%.2f%% of 100%%)\n", formatBytes((float)$storageUsed), $poolUsedPct);
echo str_repeat('-', 65) . "\n";
printf("TOTAL Provisioned / Allocated        : %s (%.2f%% of capacity%s)\n",
    formatBytes($combinedAllocated),
    $combAllocPct,
    $combAllocPct > 100 ? ' [OVERCOMMITTED]' : ''
);
printf("  - VM Disks (volsize)               : %s\n", formatBytes($vmAllocated));
printf("  - CT Containers (refquota)         : %s\n", formatBytes($ctAllocated));
echo str_repeat('-', 65) . "\n";
printf("TOTAL Actual Space Used (VMs + CTs)  : %s (%.2f%% of 100%%)\n",
    formatBytes($combinedUsed),
    $combUsedPct
);
printf("  - Active Data (Referenced)         : %s\n", formatBytes($vmUsed + $ctUsed));
printf("  - VM Disk Consumption              : %s\n", formatBytes($vmUsed));
printf("  - CT Container Consumption         : %s\n", formatBytes($ctUsed));
echo str_repeat('=', 65) . "\n\n";
?>

 

[LnxBil]

Nice idea, maybe write it in something that does not require additional programs to be installed? Perl is very similar to PHP and already installed, an AI wil help you convert it and it should be only minor syntax changes as far as I can tell.

You should not compare it to 100%, which is not reachable in ZFS. The most I saw was 93% and it was stuck then, some got stuck at 92%. You will have performance degradation starting at 85% (sometimes lower). IMHO, that should be the comparable target fill rate.

 

[Deepen Dhulla]

Yes, that is correct. Keeping ZFS pool utilization at 85% or lower is a good practice. Personally, I prefer to keep it between 60% and 70% to ensure there is enough overhead for snapshot backups in production.
Regarding the code, I generally prefer PHP for its speed , but it can certainly be converted to Perl if needed or any other language too.