Where is my SSD space going?

[Istria]

Hello all,

I'm new to proxmox and to hypervisors in general.
I have a HP T630 Thin Client (4x2GHz + 4GB DDR4 + 32GB SSD) that I installed proxmox on.

I'm wondering where I am losing my precious GBs along the way:

- During install, I chose a maxroot of 6GB and 2GB for swap. It showed a 29GB disk.
- I created 2 VM's: Home Assistant OS and Openmediavault. Have not used them yet. Fresh install + updates.
- Under Disks/LVM, I see my partition /dev/sda3 with 31.5GB. It says 88% full with 3.9GB free.
- Under LVM-Thin, I see 1 partition/volume of 16,9GB that is available for my VMs.
- Under "local", it says 2,9GB/6,25GB used.
- Under "local-lvm", it says 13GB/16,9GB used.

6,25GB root + 2GB swap + 16,9GB LVM-Thin = 25,15GB. Where did the remaining 6,5GB go?

Thanks in advance.

 

[oguz]

hi,

please check the outputs from:

df -h
lsblk -f

 

[Istria]

Here are the outputs. Thanks for the help.

EDIT: Output is in my next post below.

 

[oguz]

could you paste it here in [code][/code] tags for better visibility?

 

[Istria]

could you paste it here in [code][/code] tags for better visibility?

I will. It's so hard using a phone. I will be at my laptop in 10min.

 

[Istria]

root@proxmox:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
udev                  1.8G     0  1.8G   0% /dev
tmpfs                 364M  1.1M  363M   1% /run
/dev/mapper/pve-root  5.9G  3.3G  2.3G  60% /

root@proxmox:~# lsblk -f
NAME FSTYPE FSVER LABEL UUID
FSAVAIL FSUSE% MOUNTPOINT
sda

├─sda1
│

├─sda2
│    vfat   FAT32       FEC9-FE11

└─sda3
     LVM2_m LVM2        hgBVip-H3EW-EWDK-erOP-dQSt-s3eO-yhtcHQ

  ├─pve-swap
  │  swap   1           89a7ad6f-9397-4d6b-9196-9484ad3246ca
               [SWAP]
  ├─pve-root
  │  ext4   1.0         7afd39ef-6c62-461b-a975-cc7e311560b4
   2.3G    56% /
  ├─pve-data_tmeta
  │

  │ └─pve-data-tpool
  │

  │   ├─pve-data
  │   │

  │   ├─pve-vm--101--disk--1
  │   │

  │   ├─pve-vm--101--disk--2
  │   │

  │   ├─pve-vm--102--disk--0
  │   │

  │   ├─pve-vm--102--disk--1
  │   │

  │   └─pve-vm--105--disk--0
  │

  └─pve-data_tdata


    └─pve-data-tpool


      ├─pve-data
      │

      ├─pve-vm--101--disk--1
      │

      ├─pve-vm--101--disk--2
      │

      ├─pve-vm--102--disk--0
      │

      ├─pve-vm--102--disk--1
      │

      └─pve-vm--105--disk--0


root@proxmox:~#

 

[oguz]

thanks. and what do you see with fdisk -l /dev/sda ?
what about the size of the VM disk images? you can check the logical volumes with lvs -a

 

[Istria]

Of course:

root@proxmox:~# fdisk -l /dev/sda
Disk /dev/sda: 29.82 GiB, 32017047552 bytes, 62533296 sectors
Disk model: PSS9N032GA27MC2
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 726DC5BA-CD68-4601-85ED-8DD142586617

Device       Start      End  Sectors  Size Type
/dev/sda1       34     2047     2014 1007K BIOS boot
/dev/sda2     2048  1050623  1048576  512M EFI System
/dev/sda3  1050624 62533262 61482639 29.3G Linux LVM
root@proxmox:~# lvs -a
  LV              VG  Attr       LSize  Pool Origin Data%  Meta
%  Move Log Cpy%Sync Convert
  data            pve twi-aotz-- 15.69g             77.35  2.16

  [data_tdata]    pve Twi-ao---- 15.69g

  [data_tmeta]    pve ewi-ao----  1.00g

  [lvol0_pmspare] pve ewi-------  1.00g

  root            pve -wi-ao----  6.00g

  swap            pve -wi-ao----  2.00g

  vm-101-disk-1   pve Vwi-aotz--  4.00m data        14.06

  vm-101-disk-2   pve Vwi-aotz-- 32.00g data        9.58

  vm-102-disk-0   pve Vwi-aotz-- 10.00g data        33.96

  vm-102-disk-1   pve Vwi-aotz--  6.00g data        94.60

  vm-105-disk-0   pve Vwi-aotz--  4.00g data        0.00

root@proxmox:~#

 

[Dunuin]

You really should think about getting a SSD and RAM upgrade. 6GB isn't much for PVE. I personally wouldn't use less than 16GB for maxroot because sooner or later your logs and old packages will add up and you run out of space. Would be very annoying to manually delete old packages and logs all the time.
And only 4GB RAM won't allow you to do much with your machine. Keep in mind that RAM overprovisioning isn't really working.

 

[Istria]

You really should think about getting a SSD and RAM upgrade. 6GB isn't much for PVE. I personally wouldn't use less than 16GB for maxroot because sooner or later your logs and old packages will add up and you run out of space. Would be very annoying to manually delete old packages and logs all the time.
And only 4GB RAM won't allow you to do much with your machine. Keep in mind that RAM overprovisioning isn't really working.

I have an extra 4GB RAM on the way. SSD I was hoping to use the original and put the virtual drives on an external hdd.

Edit: Correction, I meant to use the 32 GB internal drive for both. But I didn't take into account the proxmox root would keep growing on its own. Thought I could get away with 32GB for 3 Linux VMs.
I'm on the lookout for a cheap used 128GB ssd. But in the meantime, I might try to just run 1 OS with docker containers. Hopefully that will use less resources and remember proxmox for when I have a better machine.

It is still not completely clear to me where the missing disk space went. If I add up all the sizes in the shell output, I still don't get to my 31,5GB.

 

[Istria]

Alright. An update from my side.

I added a second M2 128GB ssd to the machine. So now I have 1x 32GB + 1x 128GB. I guess the best thing is now to use the whole 32GB drive for root (local). And the whole 128GB as the local-lvm to store the VMs on.
This was not an option on install. So I:
- Installed proxmox on the 32GB SSD, selecting 2GB swap and nothing else. It resulted in 8GB PVE-root and 16GB of lvm-thin "data".
- Deleted the lvm-thin "data" volume/partition.
- Expanded the PVE-root partition to max remaining size using 2 commands I found on this forum (in German).
- Created a new lvm-thin using the 128GB ssd names "data".

1. Result seems like I want to. Although I'm not sure this is the best way to go. For example, instead of creating a lvm-thin volume, I could have also created a directory for the 128GB drive. It seems to have the same result, with the additional benefit of being able to upload ISO's to it as well. I'm not sure which is the better choice. The ISO's fit on the 32GB PVE-root, so to keep it as much the same as a fresh proxmox install, I used PVE-thin. But please correct me if there are reasons to use directory instead.

2. The 128GB SSD is way too big for my VM needs. Half it's capacity would be enough for me. Is it possible to use 64/128GB for VMs, and leave the rest for a normal ext4 partition for data storage? Or can proxmox only use the entire physical drive?

3. I noticed that swap usage of proxmox goes up when needed, but does not go down again once the RAM frees up. For example, now its showing 47% RAM usage and 450MB swap usage. Is this normal? It seems unlogical to me to keep stuff on the swap partition while fast RAM is free.

 

[Dunuin]

Alright. An update from my side.

I added a second M2 128GB ssd to the machine. So now I have 1x 32GB + 1x 128GB. I guess the best thing is now to use the whole 32GB drive for root (local). And the whole 128GB as the local-lvm to store the VMs on.
This was not an option on install. So I:
- Installed proxmox on the 32GB SSD, selecting 2GB swap and nothing else. It resulted in 8GB PVE-root and 16GB of lvm-thin "data".
- Deleted the lvm-thin "data" volume/partition.
- Expanded the PVE-root partition to max remaining size using 2 commands I found on this forum (in German).
- Created a new lvm-thin using the 128GB ssd names "data".

The PVE installer will only partition/format your boot disks. You can then add more disks later using the webUI or CLI. And you don't need to manually resize the partitions. Look here for the paragraph "Advanced LVM Configuration Options".

1. Result seems like I want to. Although I'm not sure this is the best way to go. For example, instead of creating a lvm-thin volume, I could have also created a directory for the 128GB drive. It seems to have the same result, with the additional benefit of being able to upload ISO's to it as well. I'm not sure which is the better choice. The ISO's fit on the 32GB PVE-root, so to keep it as much the same as a fresh proxmox install, I used PVE-thin. But please correct me if there are reasons to use directory instead.

A directory storage got more overhead because there is an additional filesystem. So using LVM-Thin your VMs should be faster and SSD should last longer.

2. The 128GB SSD is way too big for my VM needs. Half it's capacity would be enough for me. Is it possible to use 64/128GB for VMs, and leave the rest for a normal ext4 partition for data storage? Or can proxmox only use the entire physical drive?

I think the webUI can only work with complete disks. But you can of cause use the CLI to manually create it as you like.

3. I noticed that swap usage of proxmox goes up when needed, but does not go down again once the RAM frees up. For example, now its showing 47% RAM usage and 450MB swap usage. Is this normal? It seems unlogical to me to keep stuff on the swap partition while fast RAM is free.

Thats normal. You don't want any feedback loops. If you don't want your PVE to swap that much you could lower your swappiness.