LVM confusion on KVM host after modifying the guest

hverbeek

Member
Feb 14, 2011
40
1
8
I just experienced a weird problem on my Proxmox PVE Cluster. I have 10 or so KVMs running on a PVE host (2.6.32-4-pve). The VMs use virtio block devices that get mapped to logical volumes on the host.

Just now, in order to extend diskspace in one KVM guest (id=109, name=rio), I

  • added an additional virtio-LVM-backed disk from the web-interface
  • stopped and started the guest (/dev/vdb appeared)
  • extended the logical volume inside the guest
    • pvcreate /dev/vdb
    • vgextend rio /dev/vdb
    • lvresize -L 10GB /dev/rio/root
    • resize2fs /dev/rio/root
That worked fine. Only suddenly I get errors on the host, when doing anything involving storage (pve-manager shows HTTP 500, error involving /sbin/vgs). When I run 'vgdisplay' on the host, I see errors referring to the VG of the *guest*:

Code:
london:~# vgdisplay
  --- Volume group ---
  VG Name               mysql
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               20.00 GB
  PE Size               4.00 MB
  Total PE              5119
  Alloc PE / Size       5119 / 20.00 GB
  Free  PE / Size       0 / 0   
  VG UUID               PLq0fS-64UY-0WnA-aHcd-f25C-x1W6-zX8JHm
   
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Volume group "rio" not found
  --- Volume group ---
  VG Name               vg0
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  150
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                13
  Open LV               11
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               4.09 TB
  PE Size               4.00 MB
  Total PE              1072903
  Alloc PE / Size       664576 / 2.54 TB
  Free  PE / Size       408327 / 1.56 TB
  VG UUID               PkBfxk-9sKc-xvXm-mfHl-npEs-fBmC-XKrfPM

What is happening here? The device with the "missing UUID" is the virtio-device that KVM Guest rio sees as its initial physical volume.

Also, the volume group 'mysql' does not belong to the host, it belongs to one of the guest VMs... strange.

At the moment I cannot start any VMs because of this error or modify any storage settings. Will a reboot of the KVM Host resolve this problem? Can I manually resolve this (even better)?

Thanks for your help!!
 
I guess you should not create a VG directly on the virtual device. Instead partition the disk and create the VG on the partition.
 
is the output of pvdisplay on the host correct?
Code:
london:~# pvdisplay
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  --- Physical volume ---
  PV Name               /dev/dm-12
  VG Name               mysql
  PV Size               20.00 GB / not usable 4.00 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              5119
  Free PE               0
  Allocated PE          5119
  PV UUID               ZcE2Ez-Y5uJ-Esje-N1fb-IQww-q85u-rOyReA
   
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Volume group "rio" not found
  Skipping volume group rio
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Volume group "rio" not found
  Skipping volume group rio
  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               vg0
  PV Size               4.09 TB / not usable 1.81 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              1072903
  Free PE               408327
  Allocated PE          664576
  PV UUID               dHz9Wl-AdAs-07mZ-qifL-CMHf-yv7J-z5IA1s
What shows vgscan and lvdisplay?

Code:
london:~# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "mysql" using metadata type lvm2
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Couldn't find device with uuid '7RzjNo-PSq0-NjEc-EFGf-luil-jmA3-c6nGrW'.
  Couldn't find all physical volumes for volume group rio.
  Volume group "rio" not found
  Found volume group "vg0" using metadata type lvm2
lvdisplay output is the same - amongst the expected/correct listing of all lvols, it shows the same errors as above.

I noticed that on the host, I have /dev/vg0 and /dev/mysql, but no /dev/rio. Should I try 'vgscan --mknodes' ?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!