Local-lvm grow up every day

[RobertusIT]

Hi,

I'm freaking about it, local-lvm increase every day and i don't know why.

I have only a machine in use, Home Assistant, and i make backup ( snapshot ), in a USB mounted, so local-lvm should be not touched, but grow up.

And i also have 56GB that proxmox decided without any input by me, to use for root, that are useless, why i can't decided when i installed it ? How can decrease easily root disc? and also local-lvm to avoid this issue that grow up?

 

[dylanw]

And i also have 56GB that proxmox decided without any input by me, to use for root, that are useless, why i can't decided when i installed it ?

There's a 'maxroot' parameter in the installer (for lvm configuration), which sets the maximum value of the root volume (see section 2.3.1 of the admin guide [1]).

How can decrease easily root disc?

To shrink an LVM partition, you need to unmount the filesystem. Given that it's the root file system, this would require booting from a live usb and reducing it from there. If you want to try this, there are guides online, but I would first recommended carrying out backups of any important data.

I'm freaking about it, local-lvm increase every day and i don't know why.

Could you post the output of lvs?

 

[RobertusIT]

C'è un parametro 'maxroot' nell'installer (per la configurazione di lvm), che imposta il valore massimo del volume di root (vedi sezione 2.3.1 della guida all'amministratore [1]).


Per ridurre una partizione LVM, è necessario smontare il filesystem. Dato che è il file system root, ciò richiederebbe l'avvio da un live USB e la riduzione da lì. Se vuoi provare, ci sono guide online, ma ti consiglio prima di eseguire il backup di tutti i dati importanti.


Potresti postare l'output di lvs?

root@NUC-i3:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data pve twi-aotz-- 179.71g 7.62 1.26
root pve -wi-ao---- 25.00g
swap pve -wi-ao---- 8.00g
vm-100-disk-0 pve Vwi-a-tz-- 4.00m data 0.00
vm-100-disk-1 pve Vwi-a-tz-- 32.00g data 42.82

 

[dylanw]

Have you removed vm-101 since your original post? From the output of lvs I see that only vm-100 is accounted for, which takes up roughly 13GB or 12GiB (32g * 42.84%). In general, total usage can be calculated with: "LSize" * "Data%"
As far as the growing data is concerned, this could simply be due to the fact that you do not have Discard [1] enabled on the virtual drives. Without this, the guest won't tell the host that it has deleted data, so the host won't dynamically shrink it. To enable discard on your drives, open the "Hardware" panel of a VM, select the drive, check the "discard" box, and reboot the VM from the PVE host.

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_hard_disk_discard

 

[RobertusIT]

Have you removed vm-101 since your original post? From the output of lvs I see that only vm-100 is accounted for, which takes up roughly 13GB or 12GiB (32g * 42.84%). In general, total usage can be calculated with: "LSize" * "Data%"
As far as the growing data is concerned, this could simply be due to the fact that you do not have Discard [1] enabled on the virtual drives. Without this, the guest won't tell the host that it has deleted data, so the host won't dynamically shrink it. To enable discard on your drives, open the "Hardware" panel of a VM, select the drive, check the "discard" box, and reboot the VM from the PVE host.

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_hard_disk_discard

Is already with discard flag, but grow up anyway.

How is possible?

Need to enable TRIM ( how ) or something?

 

[Dunuin]

You need to enable discard for the virtual disk and also need to configure every guest to use discard. Windows should use discard/trim by default. Atleast if you enable "SSD emulation" so Windows thinks the virtual disk is a SSD and not a HDD. For Linux you need to tell it to use it by for example creating a cron with "fstrim -a" or adding "discard" to the mounting options of each partition in the guests fstab.

Also make sure you got no old snapshots. As long as you got snapshots, nothing newer than the oldest snapshot gets deleted and your virtual disk will grow and grow.

 

[RobertusIT]

You need to enable discard for the virtual disk and also need to configure every guest to use discard. Windows should use discard/trim by default. Atleast if you enable "SSD emulation" so Windows thinks the virtual disk is a SSD and not a HDD. For Linux you need to tell it to use it by for example creating a cron with "fstrim -a" or adding "discard" to the mounting options of each partition in the guests fstab.

Also make sure you got no old snapshots. As long as you got snapshots, nothing newer than the oldest snapshot gets deleted and your virtual disk will grow and grow.

Is already with discard flag, but grow up.

fstrim -a in home assistant shell, decrease, but how can i add a crontab? isn't installed in home assistant os

 

[Dunuin]

Crontab can normally be edited by running crontab -e or manually editing the file "/etc/crontab".

 

[RobertusIT]

Crontab can normally be edited by running crontab -e or manually editing the file "/etc/crontab".

Yes, i know, but in Home Assistant operation system, there isn't crontab installed and i didn't know if is safe to install it.
I need to search how to

 

[Dunuin]

There has to be some kind of tool to automate tasks. It doesn't needs to be crontab, any tools will do as long as it can run a command at a daily base.

 

[RobertusIT]

There has to be some kind of tool to automate tasks. It doesn't needs to be crontab, any tools will do as long as it can run a command at a daily base.

But only discard in proxmox, doesn't have effect, why needed also fstrim -a?

 

[dylanw]

There has to be some kind of tool to automate tasks. It doesn't needs to be crontab, any tools will do as long as it can run a command at a daily base.

I believe Home Assistant uses systemd, which should also include services for fstrim. You can check systemctl status fstrim.timer to see if it's enabled already, and enable it if not.

But only discard in proxmox, doesn't have effect, why needed also fstrim -a?

Trim operations are used by a system to discard unused blocks. The discard option allows the proxmox host to see unused blocks by the guest, in order to reclaim the space. Thus the guest has to run trim, in order for the host to know what it can reclaim.

 

[RobertusIT]

I believe Home Assistant uses systemd, which should also include services for fstrim. You can check systemctl status fstrim.timer to see if it's enabled already, and enable it if not.



Trim operations are used by a system to discard unused blocks. The discard option allows the proxmox host to see unused blocks by the guest, in order to reclaim the space. Thus the guest has to run trim, in order for the host to know what it can reclaim.

From Home Assistant's console in proxmox, i can see that.

# systemctl status fstrim.timer
● fstrim.timer - Discard unused blocks once a week
Loaded: loaded (/usr/lib/systemd/system/fstrim.timer; enabled; vendor preset: enabled)
Active: active (waiting) since Mon 2021-07-26 19:13:39 UTC; 2 days ago
Trigger: Mon 2021-08-02 00:53:46 UTC; 3 days left
Triggers: ● fstrim.service
Docs: man:fstrim

Jul 26 19:13:39 homeassistant systemd[1]: Started Discard unused blocks once a week.


But if i use putty and login into Home Assistant:



I'm a little confused how to, or maybe is already enable this trim, but grow up anyway local-lvm



Note. i get for now manually fstrim -a, so you can see some decrease for that in this graph

 

[dylanw]

But if i use putty and login into Home Assistant:

I have no idea why systemctl isn't found from the console in putty, since it seems you've logged into the root account. But in any case, from the console in PVE, it seems the timer should be running.
Does the usage from home assistant cap off any point, or does it continue to grow over time? Maybe you could use a tool like iotop [1] to monitor activity on the disk, and track processes which are making a lot of writes.

[1] https://linux.die.net/man/1/iotop

 

[RobertusIT]

I have no idea why systemctl isn't found from the console in putty, since it seems you've logged into the root account. But in any case, from the console in PVE, it seems the timer should be running.
Does the usage from home assistant cap off any point, or does it continue to grow over time? Maybe you could use a tool like iotop [1] to monitor activity on the disk, and track processes which are making a lot of writes.

HA in always running, and local-lvm grow up every day,
For example:
local-lvm 24/07 11:46 17.27GB
local-lvm 29/07 14:21 21.79GB

And between these date, there are some days where i shutdown HA, so can be more larger than it.

I really don't understand why happend. In Vsphere i didn't have this issue. I setup a storage and vm use it, can't over use other space.

Anyway i really want to use proxmox and i wish to figure out

 

[dylanw]

Definitely strange, but as I said, using a tool like iotop [1] to monitor the situation could be best. In addition, check for files that tend to be rotated out often, such as log files, in case some process is writing aggressively to them. It could be a case that a log file grows rapidly, then is later compressed, which is why fstrim is able to free up so much space by the time it's run.

[1] https://linux.die.net/man/1/iotop

 

[iT4RZ4N]

HA in always running, and local-lvm grow up every day,
For example:
local-lvm 24/07 11:46 17.27GB
local-lvm 29/07 14:21 21.79GB

And between these date, there are some days where i shutdown HA, so can be more larger than it.

I really don't understand why happend. In Vsphere i didn't have this issue. I setup a storage and vm use it, can't over use other space.

Anyway i really want to use proxmox and i wish to figure out

Did you ever figure why it keeps growing? I am having the same issue.

 

[piton]

I have similar issue.
2 drives 128GB (set 32GB) for OS and 512GB (set 480GB) for data.
LVM 512GB grow up every day now is 99% what will happen when reach 100%?

 

[piton]

OK I found the solution, discard option must be set on hard drive and after that fstrim works fine.