Backup size is bigger than real used space

V

vawaver

Member
Nov 14, 2020
21
2
23
51
Good evening,

I would like to ask for help.
Ubuntu server 20.04.2 + Nextcloud (via snap) is installed on one of my VMs.

There is used 322 GB of space (I have 2.8 TB disk space for this VM).

I created a backup job on my Datacenter -> whole VM is saved to my external USB drive.

And here is my problem.
The size of the backup is actually cca 462 GB is on my USB drive

So the difference is over 130 GB between the used space on VM and the backup file (vma.zsf).

I would like to know why? Is it possible to fix it?
Thank you.

My version - Proxmox 6.4.8 VE
 
Are you using zfs with raidz1/2/3? If yes your volblocksize migut he too low so that you get overhead because of bad padding.
 
Last edited:
Yes, it is ZFS and RAID1.
I used default values during the installation process.
What should I do?
Thank you for your answer.
 
Raid1 (= mirror) or raidz1 (=raid5)?

If it is Raidz1 how many drives are you using with that pool and what ashift value did you choose?
 
Last edited:
I have created RAID1 -> mirroring the first HDD to the second HDD (both hard drives are the same)
I use 2x 4TB hard drives.
 
Hmm... a few weeks ago everything was working fine but after some updates on Proxmox, it changed.
Reason unknown.
Any other idea?
 
avw said:
Does running fstrim -av reduce the size of the backup? It might only be available when using VirtIO SCSI and virtual SCSI disks.
Click to expand...
Thank you for your answer.
Unfortunately, I am not an advanced user of Proxmox, a beginner only.
That is why I do not exactly understand your reply.
I attached a screenshot of my VM configuration.
nxtcl.png
Any idea is welcome.
 
The backup is based on disk blocks. Deleted files can still take up disk blocks and therefore the backup could be larger than the currently used space. For Linux, use the fstrim command to signal to the underlying storage that those blocks are actually free. Please run fstrim -av (as root or use sudo) in a console inside the VM, and check if the next backup is smaller. (You are using VirtIO SCSI and virtual SCSI disks, so the fstrim command should be supported.)
 
  • Like
Reactions: spirit
Thank you for your reply.
Now I understand you perfectly! :)
Your explanation is easy to understand.
I gonna try your solution and let you know about the result.
 
vawaver said:
Thank you for your reply.
Now I understand you perfectly! :)
Your explanation is easy to understand.
I gonna try your solution and let you know about the result.
Click to expand...
If that works you should add "discard" to the mounting options in your guests /etc/fstab or better add a daily/weekly "fstrim - a" to your guests /etc/crontab so you don't need to it manually.
 
Backup ended after 4 hours.... filesize 462 GB :confused:
No change.
It did not help.
Another idea?
 
Code: 
tony@nxtc:~$ sudo df -h
[sudo] password for tony:
Filesystem                         Size  Used Avail Use% Mounted on
udev                               1.9G     0  1.9G   0% /dev
tmpfs                              348M  1.1M  347M   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv  2.7T  321G  2.3T  13% /
tmpfs                              1.6G     0  1.6G   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                              1.6G     0  1.6G   0% /sys/fs/cgroup
/dev/sda2                          976M  203M  707M  23% /boot
/dev/loop1                          56M   56M     0 100% /snap/core18/2066
/dev/loop0                          56M   56M     0 100% /snap/core18/1997
/dev/loop2                          71M   71M     0 100% /snap/lxd/19647
/dev/loop3                          68M   68M     0 100% /snap/lxd/20326
/dev/loop4                         218M  218M     0 100% /snap/nextcloud/27920
/dev/loop5                         218M  218M     0 100% /snap/nextcloud/28088
/dev/loop6                          33M   33M     0 100% /snap/snapd/11841
/dev/loop7                          33M   33M     0 100% /snap/snapd/12057
tmpfs                              294M     0  294M   0% /run/user/1000

Code: 
tony@nxtc:~$ sudo fstrim -av
/boot: 769.4 MiB (806809600 bytes) trimmed on /dev/sda2
/: 2.4 TiB (2612910456832 bytes) trimmed on /dev/mapper/ubuntu--vg-ubuntu--lv
 
You are using LVM inside the VM. It might interfere with discard/trim, or it might contain old data that cannot be trimmed. Would it be possible to reinstall without LVM or add another virtual disk without using LVM and copy all partitions?
EDIT: What are you using as underlying storage? ZFS (which supports discard/trim) or LVM or something else?
 
Last edited:
I am afraid that I will lose my data because there is my Nextcloud server.
I am thinking about what kind of old data is "hidden" and how can I find it and remove it manually,

There are some old snaps of Nextcloud I suppose as well.
 
Last edited:
Maybe it is not what I think it is. Maybe it is some LVM thing. Are you using LVM thin provisioning in the VM?
Can you try a backup with LZO to see what size it is? Then we know it is not a ZSTD thing.
Is Discard enabled on the virtual disk (see section 10.2.4 of the manual)? Maybe try enabling SSD emulation?
 
Thank you for your reply. A lot of new ideas from you. :cool:

Are you using LVM thin provisioning in the VM? - I do not know how to check it.
Can you try a backup with LZO to see what size it is? - I will try it. It is a great suggestion.
Is Discard enabled on the virtual disk? - Of course, this was disabled. :cool:
Maybe try enabling SSD emulation? -I gonna try all the things above first.

I will let you know.
 
So I found a solution for me now.
Compression ZSTD (fast and good)
Mode: Stop

Settings for HDD in the virtual machine
Discard - Enable
SSD Emulation - Enable
Backup - Enable

Oťher settings are disabled.

I am back to 300 GB now from 460 GB.

Thank you once again.
 
You must log in or register to reply here.
Top Bottom