Locate & Identify VM & Template storage usage

[zero269]

I am trying to figure out how to find the file sizes that my VMs and Containers are using.

For example, in Windows, I run VMs in VMware Workstation 16. I can easily go to the VM folder and check the size of any VM; such as 25 GB, 87 GB, etc. I can easily tell the difference between a FULL clone and a Linked Clone and compare their file size differences.

I cannot do this with Proxmox. I've searched endlessly trying to figure out where this information is stored within the Linux File System. The web interface for Proxmox does not contain this information at all. I've looked in every nook and cranny, and it doesn't exist. I'm exploring the File System using the Shell for the Proxmox server (node). I am not running any Linux VMs or Containers at this time. There is no Path/Target for those LVM-Thin volumes as seen below. I checked the Directory, but I'm not storing VMs/Containers on that drive. Only ISOs and Container Templates. However, I don't see those files either.

I currently have only one VM that I created today for Windows Sever 2022 Datacenter. I turned it into a Template and then created a Linked Clone off of that Template. I'm just experimenting with Proxmox and playing with the various feature sets to help determine if running a Type 1 Hypervisor like Proxmox will prove beneficial to me.

I'm trying to figure out what storage saving benefits these techniques offer in Proxmox. I've read that the templates are smaller than a VM. However, those statements do not help me identify exactly what those savings are in actual BYTES on My Proxmox for my VMs and Containers.

I'm using the default Proxmox setup (ext4), and only added the 2nd physical NVMe drive as a secondary "local_lvm2" storage drive where I created both my Windows Server VM, that I turned into a Template; as well as the Linked Clone from that Template.

I am running Proxmox VE v7.0 -8.

Where in the Linux File System are these VMs and Containers stored exactly, and how do I find the size of storage each of them are using?


Background info:

• I am new to Proxmox with limited experience with Linux. However, I know my way around the command line.
• I have 15+ years of professional IT experience, which includes Windows administration and deployments.
• I've poked around the file system in Proxmox using the default root user account, but I can't find what I'm looking for. I've wasted hours trying to understand the different file systems, and hierarchy, but nothing helpful.

Any help would be greatly appreciated.

 

[dcsapak]

Where in the Linux File System are these VMs and Containers stored exactly, and how do I find the size of storage each of them are using?

this mostly depends on the storage you use for the images,
for example, the 'local' storage is of type 'directory' and uses 'normal' files. the path of the 'local' storage is /var/lib/vz

'local-lvm' is an LVM-Thin storage, this is a block-based (not file-based) storage. To check the contents of a lvm-thin storage you can use the cli tool 'lvs' or 'lvdisplay'

this and more is all explained in detail in the offical docs: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_storage

note, that the difference between 'full' and 'linked' clones also strongly depends on the storage subsytem used, and cannot
only be related via the size (e.g in lvm-thin, the connection between volumes are very loose, even for linked clones)

 

[zero269]

this mostly depends on the storage you use for the images,
for example, the 'local' storage is of type 'directory' and uses 'normal' files. the path of the 'local' storage is /var/lib/vz

'local-lvm' is an LVM-Thin storage, this is a block-based (not file-based) storage. To check the contents of a lvm-thin storage you can use the cli tool 'lvs' or 'lvdisplay'

this and more is all explained in detail in the offical docs: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_storage

note, that the difference between 'full' and 'linked' clones also strongly depends on the storage subsytem used, and cannot
only be related via the size (e.g in lvm-thin, the connection between volumes are very loose, even for linked clones)

Hello dcsapak,

First let me say thank you for the response and advice. Although I was able to find some useful information as it relates to finding the actual storage usage for VMs and Templates, I'll admit it is a bit confusing, inconsistent and rather inconvenient when compared to my experience with Windows and VMware products.

This is what my Node >Disks look like for reference:

​

I was able to locate my VM (ID 101) here: /dev/local-lvm2, but I am unable to list their sizes. You stated that LVM storage is block-based and not file-based which may explain why the size shown is 0.

​

You recommended the lvs and lvdisplay commands:

lvdisplay /dev/local-lvm2 produces a lot of information, but nothing relevant to my needs at this time.
lvs /dev/local-lvm2 produces some limited but useful information. The Data% column is what catches my eye. However, I am not entirely sure these are the values that I'm looking for. There's also the Meta% column that appears to relate to storage usage as well. It looks like I'll need to use a calculator to add them together, unless there is a better command or command switch that I can use to give me the total storage usage and not a breakdown by types.

​

What I know:
• ID 100 is my Template (Windows Server 2022)
• ID 101 is my Linked-Clone from my Template

1. The 1st entry is blank. It's my Template (ID 100), and should not have a value of 0. According to that link you provided:
   • data "This volume uses LVM-thin, and is used to store VM images." Given that this is a Template, the actual storage usage must be stored somewhere else. I can't find anything online so far that tells me where.
2. The 2nd entry appears to be my 2nd physical drive in the laptop.
   • I am not sure where that 11.30 Data% value is coming from, considering "local-lvm2" represents a storage drive, and not a VM (or a Template).
3. The 3rd entry represents the first virtual drive in the Linked-Clone VM (ID 101).
4. The 4th entry represents the second virtual drive in the Linked-Clone VM (ID 101).

The last two entries are part of the same VM. This tells me that Proxmox/Linux wants to only show me the data in a scattered way. Although I'll admit I do not fully understand the Linux File System, its not that difficult to find file and folder sizes in a traditional Linux OS.

I feel like the Data% value of 11.30 should be listed for the 1st entry which is a Template (Windows Server), and that size appears to be consistent with one from my VMware Workstation. The first two entries are confusing and makes me second guess the data presented. If I go down this path, and create a handful of Windows OS Templates, followed by a variety of both FULL and LINKED clones, I am not confident that I will be able to track the storage usage that each one is using.

There has to be away to find the total storage usage for a given VM and a given Template.

For example, in Windows, I just look at the VM folder Properties and I know everything I need to know to help me make necessary decisions on VM creations and management. The left image shows a full VM (12.4/12.4 GB), while the one on the right is a Linked-Clone (5.09/3.67 GB) from the Source image on the left. This is the size of the Linked-Clone in the OFF state. The third image shows the size (9.23/7.83 GB) when that Linked-Clone is in the Powered ON state.

​

​

I was able to find that information within just a few seconds and I know definitively how much storage these given VMs are using. This is what I am looking to find with the VM and Template in Proxmox.

Perhaps I am not using these commands to their full potential to find what I'm looking for, but I looked through the man pages (lvs, lvdisplay) and I'm not finding anything useful.

 

[zero269]

Changed thread title.
”Linux File System” was obviously too vague.

 

[dcsapak]

The Data% column is what catches my eye. However, I am not entirely sure these are the values that I'm looking for. There's also the Meta% column that appears to relate to storage usage as well.

yes, thats the information you're looking for. The Data% column says how much of its size is actually used.
(i.e. a lv with size 100G and Data% 50 would use 50G of data)

the Meta% is only visible for thinpools themselves (not logical thin volumes) and shows how much % of the metadata pool is used (thats the place where lvmthin stores its mapping, etc)

The 1st entry is blank. It's my Template (ID 100), and should not have a value of 0. According to that link you provided:

• data "This volume uses LVM-thin, and is used to store VM images." Given that this is a Template, the actual storage usage must be stored somewhere else. I can't find anything online so far that tells me where.

that is because the logical volume is not activated. lvm needs a volume to be active to show you more detailed information like its mapped size (i.e. the real usage).
while you *could* activate that lv to get the Data% column, this is disabled by default to prevent changes to templates.
to enable acitvation and activate you can use 'lvchange' (see 'man lvchange' for details)

The 2nd entry appears to be my 2nd physical drive in the laptop.

• I am not sure where that 11.30 Data% value is coming from, considering "local-lvm2" represents a storage drive, and not a VM (or a Template).

yes thats your thinpool. the Data% here shows you how much data of the pool is used

Although I'll admit I do not fully understand the Linux File System, its not that difficult to find file and folder sizes in a traditional Linux OS.

this has nothing to do with a file system. in fact if you are more comfortable with normal files, you can very well add a 'directory' based storage anytime. this will have your 'normal' disk images which have a size.
by default lvmthin is chosen because it is as flexible as disk images, but should perform better (because there is no filesystem overhead, only a simple mapping of disk regions)

There has to be away to find the total storage usage for a given VM and a given Template.

yes, but for lvmthin that information is not available with a single command, it'll only ever show the real usage in % of the size

if you use a single directory storage for you vm images, you can basically do the same as on windows and do a 'ls -lh /path/to/the/vm/images'
e.g. in case of the 'local' storage (which lies in /var/lib/vz)

you could do:

ls -lh /var/lib/vz/images/100

to get the 'virtual' sizes of the images and

du -sh /var/lib/vz/images/100/*

for the 'real' usage

just note that it gets much more complicated if you use qcow2 + snapshots, since they are a part of the file then..
(in contrast to lvmthin where the snapshots are seperate lvs)

in any case, i'd check out our documentation https://pve.proxmox.com/wiki/Storage as well as the relevant manpages (e.g. for lvmthin, type 'man lvmthin')

 

[zero269]

yes, thats the information you're looking for. The Data% column says how much of its size is actually used.
(i.e. a lv with size 100G and Data% 50 would use 50G of data)

the Meta% is only visible for thinpools themselves (not logical thin volumes) and shows how much % of the metadata pool is used (thats the place where lvmthin stores its mapping, etc)


that is because the logical volume is not activated. lvm needs a volume to be active to show you more detailed information like its mapped size (i.e. the real usage).
while you *could* activate that lv to get the Data% column, this is disabled by default to prevent changes to templates.
to enable acitvation and activate you can use 'lvchange' (see 'man lvchange' for details)


yes thats your thinpool. the Data% here shows you how much data of the pool is used



this has nothing to do with a file system. in fact if you are more comfortable with normal files, you can very well add a 'directory' based storage anytime. this will have your 'normal' disk images which have a size.
by default lvmthin is chosen because it is as flexible as disk images, but should perform better (because there is no filesystem overhead, only a simple mapping of disk regions)



yes, but for lvmthin that information is not available with a single command, it'll only ever show the real usage in % of the size

if you use a single directory storage for you vm images, you can basically do the same as on windows and do a 'ls -lh /path/to/the/vm/images'
e.g. in case of the 'local' storage (which lies in /var/lib/vz)

you could do:

ls -lh /var/lib/vz/images/100

to get the 'virtual' sizes of the images and

du -sh /var/lib/vz/images/100/*

for the 'real' usage

just note that it gets much more complicated if you use qcow2 + snapshots, since they are a part of the file then..
(in contrast to lvmthin where the snapshots are seperate lvs)

in any case, i'd check out our documentation https://pve.proxmox.com/wiki/Storage as well as the relevant manpages (e.g. for lvmthin, type 'man lvmthin')

Hello dcsapak,

Thanks for getting back to me. It sounds like I'll take your advice and just use a Directory type storage over the LVM-Thin storage. I doubt I'll notice any performance differences, but for now I really just need to get some VM's up and running so I can play around with Proxmox and get a better feel for it. I am not looking to setup anything long term. It's really just for learning Proxmox and I'll expand my techniques over time.

However, initially, my main focus was to see if there was a storage savings benefit between a Windows Server VM and a Windows Server Template. And if so, how much storage savings am I getting. Considering I can't identify the actual storage being consumed by the VM versus the Template using LVM-Thin, hopefully I can see it when I switch over to Directory storage.

I tried playing around with the lvchange, but I didn't have any success.
I first tried activating the logical volume using lvchange -ay vg/lv using guidance from redhat.com, but that didn't work.



I even tried activating the volume group as redhat.com says "You can and activate or deactivate all of the logical volumes in a volume group with the -a option of the vgchange command."



I'm not sure where to run the command from considering if I run any of the scans (pvs, vgs, lvs) from this location, they run.

So, basically, I don't have time to figure all of this out right now. With the limited time I have, I will need to revisit this down the road once I get a better handle on these varying storage complexities. This is why I never had an interest in enterprise storage. It's … boring!

Again, thank you for all your help.