[SOLVED] hard links in vm are sent as a full file

RobFantini

Famous Member
May 24, 2012
2,009
102
133
Boston,Mass
we use rsnapshot for data backups within vm's. in case you are not familiar with rsnapshot - from the man page:
Code:
      rsnapshot is a filesystem snapshot utility. It can take incremental snapshots of local and remote
       filesystems for any number of machines.
       ..
           rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the
       size of one full backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use
       of hard links, so if the file doesn't change, the next snapshot is simply a hard link to the exact same
       file.

we have 3 large rsnapshot backup vm's, each with 1TB or great disk for rsnapshot use.

for one of those:

backup to ceph-fs uses 70GiB
backup to pbs uses 1.58TiB


Still since PBS may use some rsync like code , is it possible that turning on something like rsync's -H flag could increase efficiency ? [ probably not I'd guess ] .

overall we are very pleased with PBS..
 
are you doing a host backup using proxmox-backup-client from within the VM? in that case, hardlinks should just consume metadata for the hardlinks, but not store the files multiple times.. if you use vzdump on the PVE host, than PBS does not operate on the file system level, but on the block level. maybe what you are seeing is logical size (PBS) vs compressed size (old-style vzdump to cephfs)?
 
I am using vzdump from pve host.

will look into the other option

thank you.

possibly you don't need to - PBS will only show you the logical size, not how much space the actually stored (compressed, deduplicated) chunks use. check the .chunks directory size for the current usage of a datastore.
 
that is true - after backing up the vm again, storage used at pbs before and after were close to the same.

155GB used at PBS per df and zfs list . backing the 1.5TB vm was done two times along with 20 other vm's.

so actual storage space used at pbs is remarkably low .
 
  • Like
Reactions: fabian

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!