ProxVE Cluster / Shared Storage -> Best way to resize (shrink) VM disk allocation?

[fortechitsolutions]

Hi, a 'quick' question. I've setup a modest sized ProxVE cluster, 3 nodes in an intel modular server with shared "SAS-SAN" storage between nodes for cluster:SAN VM storage pool / with LVM overlay (as per typical deployment recommendation).

I was curious, in this configuration, is there an 'easy' way to change (shrink) disk space allocation to a VM ? Since there is no 'direct' access to the underlying LVM 'filesystem' in this config .. typical under-the-hood method (qemu-img based ..) are not really accessible.

I wonder thus, are my options: To Shrink the VM Disk allocation for example,
- Inside the VM, use Guest / OS-based tool to shrink the filesystem allocation (gparted or similar)
- Power off VM, backup, restore to different storage (NFS or local storage)
- reduce the size of the .RAW or QCOW (or whatever format it is..) VM image file on your filesystem-accessible location
- if required tweak config of the VM files to indicate size of file/virtual HDD (don't think this is actually required)
- take a backup of the VM again
- do a restore of the VM, and dump it back into the LVM storage/Shared SAN storage pool. OR dump onto another local:NFS:iSCSI shared SAN storage which you have available / which is appropriate / desired as resting place for the VM.

but - basically - there is no way to do this song and dance without doing a VM backup, restore, backup, restore cycle... ?

If I am missing something obvious, a friendly hint is greatly appreciated!

Thanks,

Tim

 

[udo]

Re: ProxVE Cluster / Shared Storage -> Best way to resize (shrink) VM disk allocation

Hi, a 'quick' question. I've setup a modest sized ProxVE cluster, 3 nodes in an intel modular server with shared "SAS-SAN" storage between nodes for cluster:SAN VM storage pool / with LVM overlay (as per typical deployment recommendation).

I was curious, in this configuration, is there an 'easy' way to change (shrink) disk space allocation to a VM ? Since there is no 'direct' access to the underlying LVM 'filesystem' in this config .. typical under-the-hood method (qemu-img based ..) are not really accessible.

I wonder thus, are my options: To Shrink the VM Disk allocation for example,
- Inside the VM, use Guest / OS-based tool to shrink the filesystem allocation (gparted or similar)
- Power off VM, backup, restore to different storage (NFS or local storage)
- reduce the size of the .RAW or QCOW (or whatever format it is..) VM image file on your filesystem-accessible location
- if required tweak config of the VM files to indicate size of file/virtual HDD (don't think this is actually required)
- take a backup of the VM again
- do a restore of the VM, and dump it back into the LVM storage/Shared SAN storage pool. OR dump onto another local:NFS:iSCSI shared SAN storage which you have available / which is appropriate / desired as resting place for the VM.

but - basically - there is no way to do this song and dance without doing a VM backup, restore, backup, restore cycle... ?

If I am missing something obvious, a friendly hint is greatly appreciated!

Thanks,

Tim

Hi,
you must first reduce the space inside the VM (or you have data lost). After that, you can use lvresize to reduce the size of an lv... (and after stop/start of the VM you must change the partition-table...).

Be sure to have an valid backup!

Udo

 

[fortechitsolutions]

Re: ProxVE Cluster / Shared Storage -> Best way to resize (shrink) VM disk allocation

Hi Udo, sorry for confusion my post may have created.

- My setup is similar to ProxVE setup docs, http://pve.proxmox.com/wiki/Intel_Modular_Server#Pool2:_Shared_LVM_storage_for_KVM_guests
- ie, I have got a "Shared LVM Storage for KVM Guests".
- so the VM itself has no LVM (or if it does - that is separate layer / issue entirely) - but for purpose of discussion here, assume the guest VM has no LVM.
- rather, LVM is implemented as part of Proxmox, Shared storage pool / SAN storage.
- and in this config, I cannot see into the "shared lvm storage pool' as the root user when on SSH session on any of the proxmox hosts
- in contrast, looking at local or NFS based storage - I have "filesystem visibility" and expect to see files and so forth associated with VM Guests when I have VMs that use NFS backing data storage for example.

So - in that context - does my question make better sense ?

Thanks for your reply and assistance!


Tim

 

[fortechitsolutions]

Re: ProxVE Cluster / Shared Storage -> Best way to resize (shrink) VM disk allocation

Icing on cake, might as well ask. Recent (unrelated project) work with OpenStack led me to believe there may be a feature possible for LVM VM storage configuration, that allows you to enable "thin provisioning" . I wonder if there is anything analogous to this for ProxmoxVE LVM implementation, or if LVM backed/SharedSanStorage - is inherently non-thin-provision based ... just because that is how it is ? Figure I might as well check since it is more or less a related topic to my initial post:query. Thanks! - Tim

 

[udo]

Re: ProxVE Cluster / Shared Storage -> Best way to resize (shrink) VM disk allocation

Hi Tim,
if the VM-disk on an filesystem instead of an lv, you can use qemu-img to shrink the file (but need temporarly space for the big snd the shrinked one).
See man-page:

qemu-img - QEMU disk image utility

SYNOPSIS
       usage: qemu-img command [command options]

...
OPTIONS
       The following commands are supported:
...
       resize filename [+ | -]size

If the vm is on lvm you see something like this:

lvs
  LV            VG                    Attr      LSize   Pool Origin Data%  Move Log Copy%  Convert
  vm-108-disk-1 a_sas_r0              -wi-ao---  40.00g                                           
...

But again, you need to freed up the space before converting inside the VM!
If I shrink this file below 12872563200 bytes the VM-filesystem is damaged:

file vm-298-disk-1.raw 
vm-298-disk-1.raw: x86 boot sector, Microsoft Windows XP MBR (german), Serial 0xa9c8a9c8; partition 1: ID=0x7, active, starthead 1, startsector 63, 25141662 sectors, code offset 0xc0
ls -l vm-298-disk-1.raw
-rw-r--r-- 1 root root 12884901888 Jul 18  2013  vm-298-disk-1.raw
echo "512 * (25141662+63)" | bc
12872563200

Udo

 

[fortechitsolutions]

Re: ProxVE Cluster / Shared Storage -> Best way to resize (shrink) VM disk allocation

Hi Udo, thanks for the reply. Greatly appreciated.

I am learning - good! - I was not entirely aware of precisely how the LVM was being used for the shared storage; now I get it (I think).

So, on this deployment, output from "lvs" command gives me a list of all the VMs which are using LVM backed shared storage, for example:

root@prox1:~# lvs

  LV            VG      Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert
  data          pve     -wi-ao--  50.00g                                         
  root          pve     -wi-ao--  25.25g                                         
  swap          pve     -wi-ao--  12.62g                                         
  vm-100-disk-1 vmdisks -wi-ao-- 500.00g                                         
  vm-101-disk-1 vmdisks -wi-a--- 200.00g                                         
  vm-102-disk-1 vmdisks -wi-ao-- 500.00g
...etc....

So - I guess what I am unclear on: How can I resize (shrink in my case) one of these logical volumes? Is it simply a matter of using the 'lvreduce' command on the proxmox node .. presumably while the VM in question is powered off / and after the filesystem inside first has been shrunk accordingly ?

And final fun question: any idea how 'other platforms' manage to achieve some form of thin provisioning on LVM backed storage ? Such a thing would be awesome if there was a way to pull it off here ?

Thanks for your help with this,

Tim

FOOTNOTE added after initial post:
I just dug on google, and find, for example:

Redhat 6.4 LVM appears to support thin provision LVM (but specifically LVM here in past did not support thin..)
https://access.redhat.com/site/docu...ation/thinly_provisioned_volume_creation.html

and
nice discussion of thin LVM on Debian, including a comment that it was not really available at the time of this discussion below (may-2013)
http://hydra.geht.net/tino/howto/linux/lvm2/thin/

so I am guessing, maybe this is an LVM implementation feature, which may be partially or not yet fully implemented; or at least not yet present in LVM as used on ProxVE presently ? But possibly you can comment