[SOLVED] Proxmox IO-Error - LVM-Thin full ?

[tripton]

Hi guys,

Sorry if this has been asked before but I couldn't find a direct answer to my issue .

I recently started using Proxmox to run my HA VM and after a couple of weeks proxmox stopped being able to connect to my VM shortly after booting it .

I am getting io-error and after some searching I believe this is almost surely related to some issue with disk space, although my VM should have enough free space available .

Here is my HDD data :

df -h
Filesystem            Size  Used Avail Use% Mounted on
udev                  2.9G     0  2.9G   0% /dev
tmpfs                 580M   38M  543M   7% /run
/dev/mapper/pve-root   14G  2.9G   10G  23% /
tmpfs                 2.9G   43M  2.8G   2% /dev/shm
tmpfs                 5.0M     0  5.0M   0% /run/lock
tmpfs                 2.9G     0  2.9G   0% /sys/fs/cgroup
/dev/fuse              30M   16K   30M   1% /etc/pve
tmpfs                 580M     0  580M   0% /run/user/0

root@proxmox:~# lvs
  LV            VG  Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data          pve twi-aotzD- 27.77g             100.00 2.88
  root          pve -wi-ao---- 13.75g
  swap          pve -wi-ao----  5.00g
  vm-100-disk-0 pve Vwi-a-tz-- 53.00g data        52.40

root@proxmox:~# dmsetup ls --tree
pve-vm--100--disk--0 (253:6)
 └─pve-data-tpool (253:4)
    ├─pve-data_tdata (253:3)
    │  └─ (8:3)
    └─pve-data_tmeta (253:2)
       └─ (8:3)
pve-swap (253:0)
 └─ (8:3)
pve-root (253:1)
 └─ (8:3)
pve-data (253:5)
 └─pve-data-tpool (253:4)
    ├─pve-data_tdata (253:3)
    │  └─ (8:3)
    └─pve-data_tmeta (253:2)
       └─ (8:3)

root@proxmox:~# lvs
  LV            VG  Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data          pve twi-aotzD- 27.77g             100.00 2.88
  root          pve -wi-ao---- 13.75g
  swap          pve -wi-ao----  5.00g
  vm-100-disk-0 pve Vwi-a-tz-- 53.00g data        52.40

root@proxmox:~# pvs
  PV         VG  Fmt  Attr PSize  PFree
  /dev/sda3  pve lvm2 a--  55.39g 6.87g

My LVM-Thin volume is full ,is this the reason for my io-error ?

Sorry proxmox noob here, how can I go about fixing this ? Am unable to even log to my VM as it is .

Thanks !

 

[oguz]

hi,

I am getting io-error and after some searching I believe this is almost surely related to some issue with disk space, although my VM should have enough free space available .

one thing you need to understand first is thin provisioning feature of lvmthin, which allows you to allocate more space than physically available.

in pve/data you have 27.77g available. vm-100-disk-0 has 53.00g assigned (thin-provisioned) and %52.40 percent of that is used, which adds up to 27.77g. that means data is full and can't allocate anything else for your VM (results in I/O error since it's 100% used)

this means even if you see the vm disk not 100% full, that's only because you assigned a larger amount of space than physically available.

how can I go about fixing this ? Am unable to even log to my VM as it is .

* extend your available space by adding another disk or getting a bigger one
or
* clean up some space in data (which means files in your VM)

 

[tripton]

hi,



one thing you need to understand first is thin provisioning feature of lvmthin, which allows you to allocate more space than physically available.

in pve/data you have 27.77g available. vm-100-disk-0 has 53.00g assigned (thin-provisioned) and %52.40 percent of that is used, which adds up to 27.77g. that means data is full and can't allocate anything else for your VM (results in I/O error since it's 100% used)

this means even if you see the vm disk not 100% full, that's only because you assigned a larger amount of space than physically available.


* extend your available space by adding another disk or getting a bigger one
or
* clean up some space in data (which means files in your VM)

Hi, tyvm for the feedback @oguz !

I only extended the disk space after I started getting this error, originally I assigned 32G to my Raw disk and I am still a bit confused why I am running out of space since the VM should only be using around 13 out of 32Gb .

I have in the meantime increased my VM disk to 53G hoping that would fix the issue but from what I understand I should have resized my LVM as well ?

So in this case, would resizing my LVM volume manually to 53.00g do the trick ? It's a 64g ssd so I should have the space physically available

 

[tripton]

Managed to fix it by resizing LVM-Thin Volume to 32G !

Thanks

 

[oguz]

great!

please mark the thread [SOLVED] so others know what to expect as well

 

[monaco]

Managed to fix it by resizing LVM-Thin Volume to 32G !

Thanks

How did you resize it?