[SOLVED] snapshots ZFS?

[killmasta93]

Hi,
I was wondering if someone could shed some light on snap shots or backup solutions for the vm.
Currently using vzdump to backup my vms but currently doing it weekly, which takes a long time. My question is What other option is to backup the vm as a differential or incremental? I was reading about pve-zsync but i would need 2 hosts of promox? I also read about zfSnap but not sure if this would work. And lastly maybe rsync? the idea is to have daily backups as a differential so the process wont take long enough. I also read this https://ayufan.eu/projects/proxmox-ve-differential-backups/ and tried it out, but for some odd reason it did not work as i thought, as the 2nd backup took the same time as the 1st backup


Thank you

 

[fireon]

Yes i do this for vm's with pve-zsync. You can also use zfs directly. If that, target must not be an PVE, it can also be some solaris or other... but pve-zsync is the easy way for do this with the whole VM. An other thing would be BackupPC if you would like to backup only files. Nice Webinterface, and it uses rsync. For Ubuntu the packages for version 4 are here.

 

[slanbuas]

There is few way you can use to accelerate your vzdump, an easy way if you are not already and have a lot of unused CPU could be to replace gzip by pigz.
https://pve.proxmox.com/pve-docs/chapter-vzdump.html

A zfs snapshotting solutions should works. You can make it work with only one host between 2 zfs drives.

An option I use for archival is an uncompressed vzdump (zfs send could be used too) + borg backup. Since borg is a blocks based deduplicating backup solution you can backup pretty often without using much data and is pretty fast.

https://borgbackup.readthedocs.io/en/stable/

 

[fireon]

An option I use for archival is an uncompressed vzdump (zfs send could be used too) + borg backup. Since borg is a blocks based deduplicating backup solution you can backup pretty often without using much data and is pretty fast.

If you save your VM Backup on ZFS uncomrpessed (vma) and activate auf ZFS target dedup then you have the same. But you need a lot of mem an I/O.

 

[killmasta93]

@fireon thanks for the reply, really interesting the BackupPC for only files, I saw you mention pve-zsync, is there a site you can guide me so i can try it out? Do i need another proxmox machine that is also running zfs to backup the snapshots? or can snapshot it on a NAS or USB device?

@slanbuas thanks for the reply, unfortunately the CPU is being used alot so cannot use that compression mode, I was thinking something like differential Backups as the 1st one takes along time then the 2nd much quicker and etc.

Thank you

 

[CharlesErickT]

ZFS snapshot and borgbackup of the snapshot to a NAS

 

[fireon]

thanks for the reply, really interesting the BackupPC for only files, I saw you mention pve-zsync, is there a site you can guide me so i can try it out?

https://pve.proxmox.com/wiki/PVE-zsync
https://github.com/backuppc/backuppc (install version 4 on an Container of KVM, version 3 is to old)

Do i need another proxmox machine that is also running zfs to backup the snapshots? or can snapshot it on a NAS or USB device?

Not really, but with PVE as Backupserver it is much easier. If you have an other ZFSserver like solaris or zfs on ubuntu, you have not pve-sync available to install, yes you can compile it, or you use "zfs send..." with an script in cron... so it is hard do confgure out without PVE as backupserver. You have another advantage: You can start and test backups that you have done.

Yes you can also use an USBdevice, formated with ZFS.

 

[killmasta93]

Thanks for the replies, Currently i have 2 proxmox host
host 1:
Rpool the OS--Raid 1 zfs
vmdata the VMs--RAID 10 ZFS
with 32 gigs of ram
vm: windows server dhcp/ad/dns, website, email server, pfsense

host2:
Rpool the os and data---raid 1 zfs
8gigs of ram
vm: pfsense and windows server 2012r2 segundary DNS

So if the host2 has enough space usage i can daily snapshot the vms from host 1 to host 2? using pve sync?

Thank you

 

[fireon]

8gigs of ram

The host will crash in times, memory for ZFS to low. Min. 32GB

 

[killmasta93]

Thanks for the reply, as for the memory of zfs I gave it arc max of 500mb and its currently just a test server. So in theory pve sync takes a snapshot of all my vms and copies it to another host which has zfs and in theory could restore it in that host or I can just restore in the previous host?
Thank you

 

[Vasu Sreekumar]

I set c_max to 1GB, and running 10 guests per node. And ZFS work perfect for backups, and we can restore to same node.

You can also copy backup to another node and restore guest very easily.

 

[fireon]

You can also copy backup to another node and restore guest very easily.

Yes. With "zfs send".

 

[fireon]

So in theory pve sync takes a snapshot of all my vms and copies it to another host which has zfs and in theory could restore it in that host or I can just restore in the previous host?

Per VM. And yes you restore it on the backup and on the previous or an other host.

 

[killmasta93]

Thank you again for the replies over the weekend im going to give it a try to see how it goes.

Thank you

 

[fireon]

Thank you again for the replies over the weekend im going to give it a try to see how it goes.

Have FUN

 

[killmasta93]

@fireon well i finally tried pve-zsync but have tiny issue
I was trying pve-zsync from host 1 to host 2 which worked, then when i tried restoring it for some odd reason it was not located on host 2 but it was located on host 1 Then i check host 1 by running

   zfs list -H -o name -t snapshot
   vmdata/vm-100-disk-1@mine
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:33:01
   vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:34:01
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:35:01
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:36:01
   vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:37:01
   vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:38:01
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:39:01
   vmdata/vm-103-disk-2@virmin

Any ideas howcome its on host 1 the snapshots?

this is the command i ran host 1 192.168.3.252 and host 2 192.168.3.80
but before on Host 2 i needed to create a directory

 zfs create -s -V 400g vmdata/vmbackup

then on host 1 ran this

pve-zsync create --source 100 --dest 192.168.3.80:vmdata/vmbackup --verbose --maxsnap 7 --name backupsdaily

Thank you

 

[fireon]

Snapshots on both hosts, this is needed that zfs is syncing only the differences. What you mean with "it was not located on host 2" so you can't find the backup? You backuped it in here "vmdata/vmbackup" so it is not there? What say "zfs list" on host 2? You can restore it for example with:

zfs send -v rpool/home@bla1 | ssh otherhost "/usr/sbin/zfs receive otherpool/home@bla1"

Here my documentation, sorry, only german https://deepdoc.at/dokuwiki/doku.ph...:linux_zfs#zfs_snapshots_und_deren_verwaltung

 

[killmasta93]

Thanks for the reply, as for

What you mean with "it was not located on host 2" so you can't find the backup?

whats odd is that when i try to find the snapshots from host 1 which i sent it to host 2 it only appears in Host 1 when it should be in host 2.

when i ran zfs list on host 1 showed the snapshots and when i ran zfs list on host 2 it only showed the snapshots of the vm in host 2

so i reran the test this is what i did:

1) Host 1 192.168.3.252 would run

pve-zsync create --source 123 --dest 192.168.3.80:vmdata/vmbackup --verbose --maxsnap 7 --name test2

2) after it finishes the sync i would run on host 1

zfs list -H -o name -t snapshot

which i would get this which i dont understand why

vmdata/vm-123-disk-1@rep_test2_2018-03-23_19:30:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_19:45:02
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:00:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:15:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:30:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:45:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:00:01

and on host 2 which is 192.168.3.80 and ran this code i get nothing

zfs list -H -o name -t snapshot

the restore code i was using was this i would change the only thing the date

zfs send vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:00:01| ssh root@192.168.3.252 zfs receive vm/vm-123-disk-1

What does not make sense is on Host 2 I check with zfs list and i see the vmdata/vmbackup 14 gigs which it was original 64k,
This is host 2

rpool                 10.5G   888G    96K  /rpool
rpool/ROOT            3.08G   888G    96K  /rpool/ROOT
rpool/ROOT/pve-1      3.08G   888G  3.08G  /
rpool/data              96K   888G    96K  /rpool/data
rpool/swap            7.44G   895G  1.04G  -
vmdata                32.7G   866G    96K  /vmdata
vmdata/vm-100-disk-1   860M   866G   860M  -
vmdata/vm-101-disk-1  14.3G   866G  14.3G  -
vmdata/vm-101-disk-2   735M   866G   735M  -
vmdata/vmbackup       14.6G   866G    64K  -

Thank you

 

[fireon]

ok, very strange thing... please post the output of "zfs list -t snapshot" on both hosts.

 

[killmasta93]

thanks for the reply,
so host 2 i get this

root@prometheus2:~# zfs list -t snapshot
no datasets available

and on host 1 i get this

vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:30:01  2.11M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:45:01  1.47M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:00:01  1.48M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:15:01  1.22M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:19:01     0B      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:20:01     0B      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:21:01     0B      -  13.4G  -