[SOLVED] snapshots ZFS?

K

killmasta93

Renowned Member
Aug 13, 2017
958
56
68
30
Hi,
I was wondering if someone could shed some light on snap shots or backup solutions for the vm.
Currently using vzdump to backup my vms but currently doing it weekly, which takes a long time. My question is What other option is to backup the vm as a differential or incremental? I was reading about pve-zsync but i would need 2 hosts of promox? I also read about zfSnap but not sure if this would work. And lastly maybe rsync? the idea is to have daily backups as a differential so the process wont take long enough. I also read this https://ayufan.eu/projects/proxmox-ve-differential-backups/ and tried it out, but for some odd reason it did not work as i thought, as the 2nd backup took the same time as the 1st backup


Thank you
 
Yes i do this for vm's with pve-zsync. You can also use zfs directly. If that, target must not be an PVE, it can also be some solaris or other... but pve-zsync is the easy way for do this with the whole VM. An other thing would be BackupPC if you would like to backup only files. Nice Webinterface, and it uses rsync. For Ubuntu the packages for version 4 are here.
 
There is few way you can use to accelerate your vzdump, an easy way if you are not already and have a lot of unused CPU could be to replace gzip by pigz.
https://pve.proxmox.com/pve-docs/chapter-vzdump.html

A zfs snapshotting solutions should works. You can make it work with only one host between 2 zfs drives.

An option I use for archival is an uncompressed vzdump (zfs send could be used too) + borg backup. Since borg is a blocks based deduplicating backup solution you can backup pretty often without using much data and is pretty fast.

https://borgbackup.readthedocs.io/en/stable/
 
  • Like
Reactions: fireon
slanbuas said:
An option I use for archival is an uncompressed vzdump (zfs send could be used too) + borg backup. Since borg is a blocks based deduplicating backup solution you can backup pretty often without using much data and is pretty fast.
Click to expand...
If you save your VM Backup on ZFS uncomrpessed (vma) and activate auf ZFS target dedup then you have the same. But you need a lot of mem an I/O.
 
@fireon thanks for the reply, really interesting the BackupPC for only files, I saw you mention pve-zsync, is there a site you can guide me so i can try it out? Do i need another proxmox machine that is also running zfs to backup the snapshots? or can snapshot it on a NAS or USB device?

@slanbuas thanks for the reply, unfortunately the CPU is being used alot so cannot use that compression mode, I was thinking something like differential Backups as the 1st one takes along time then the 2nd much quicker and etc.

Thank you
 
killmasta93 said:
thanks for the reply, really interesting the BackupPC for only files, I saw you mention pve-zsync, is there a site you can guide me so i can try it out?
Click to expand...
https://pve.proxmox.com/wiki/PVE-zsync
https://github.com/backuppc/backuppc (install version 4 on an Container of KVM, version 3 is to old)

killmasta93 said:
Do i need another proxmox machine that is also running zfs to backup the snapshots? or can snapshot it on a NAS or USB device?
Click to expand...
Not really, but with PVE as Backupserver it is much easier. If you have an other ZFSserver like solaris or zfs on ubuntu, you have not pve-sync available to install, yes you can compile it, or you use "zfs send..." with an script in cron... so it is hard do confgure out without PVE as backupserver. You have another advantage: You can start and test backups that you have done.

Yes you can also use an USBdevice, formated with ZFS.
 
Thanks for the replies, Currently i have 2 proxmox host
host 1:
Rpool the OS--Raid 1 zfs
vmdata the VMs--RAID 10 ZFS
with 32 gigs of ram
vm: windows server dhcp/ad/dns, website, email server, pfsense

host2:
Rpool the os and data---raid 1 zfs
8gigs of ram
vm: pfsense and windows server 2012r2 segundary DNS

So if the host2 has enough space usage i can daily snapshot the vms from host 1 to host 2? using pve sync?

Thank you
 
Thanks for the reply, as for the memory of zfs I gave it arc max of 500mb and its currently just a test server. So in theory pve sync takes a snapshot of all my vms and copies it to another host which has zfs and in theory could restore it in that host or I can just restore in the previous host?
Thank you
 
killmasta93 said:
So in theory pve sync takes a snapshot of all my vms and copies it to another host which has zfs and in theory could restore it in that host or I can just restore in the previous host?
Click to expand...
Per VM. And yes you restore it on the backup and on the previous or an other host.
 
  • Like
Reactions: killmasta93
Thank you again for the replies over the weekend im going to give it a try to see how it goes.

Thank you
 
@fireon well i finally tried pve-zsync but have tiny issue
I was trying pve-zsync from host 1 to host 2 which worked, then when i tried restoring it for some odd reason it was not located on host 2 but it was located on host 1 Then i check host 1 by running
Code: 
   zfs list -H -o name -t snapshot
   vmdata/vm-100-disk-1@mine
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:33:01
   vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:34:01
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:35:01
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:36:01
   vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:37:01
   vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:38:01
  vmdata/vm-100-disk-1@rep_backupsdaily_2018-03-20_19:39:01
   vmdata/vm-103-disk-2@virmin

Any ideas howcome its on host 1 the snapshots?

this is the command i ran host 1 192.168.3.252 and host 2 192.168.3.80
but before on Host 2 i needed to create a directory

Code: 
 zfs create -s -V 400g vmdata/vmbackup

then on host 1 ran this
Code: 
pve-zsync create --source 100 --dest 192.168.3.80:vmdata/vmbackup --verbose --maxsnap 7 --name backupsdaily

Thank you
 
Snapshots on both hosts, this is needed that zfs is syncing only the differences. What you mean with "it was not located on host 2" so you can't find the backup? You backuped it in here "vmdata/vmbackup" so it is not there? What say "zfs list" on host 2? You can restore it for example with:
Code: 
zfs send -v rpool/home@bla1 | ssh otherhost "/usr/sbin/zfs receive otherpool/home@bla1"
Here my documentation, sorry, only german ;) https://deepdoc.at/dokuwiki/doku.ph...:linux_zfs#zfs_snapshots_und_deren_verwaltung
 
Thanks for the reply, as for
What you mean with "it was not located on host 2" so you can't find the backup?
Click to expand...
whats odd is that when i try to find the snapshots from host 1 which i sent it to host 2 it only appears in Host 1 when it should be in host 2.

when i ran zfs list on host 1 showed the snapshots and when i ran zfs list on host 2 it only showed the snapshots of the vm in host 2

so i reran the test this is what i did:

1) Host 1 192.168.3.252 would run
Code: 
pve-zsync create --source 123 --dest 192.168.3.80:vmdata/vmbackup --verbose --maxsnap 7 --name test2
2) after it finishes the sync i would run on host 1
Code: 
zfs list -H -o name -t snapshot
which i would get this which i dont understand why
Code: 
vmdata/vm-123-disk-1@rep_test2_2018-03-23_19:30:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_19:45:02
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:00:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:15:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:30:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:45:01
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:00:01
and on host 2 which is 192.168.3.80 and ran this code i get nothing
Code: 
zfs list -H -o name -t snapshot
the restore code i was using was this i would change the only thing the date
Code: 
zfs send vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:00:01| ssh root@192.168.3.252 zfs receive vm/vm-123-disk-1

What does not make sense is on Host 2 I check with zfs list and i see the vmdata/vmbackup 14 gigs which it was original 64k,
This is host 2
Code: 
rpool                 10.5G   888G    96K  /rpool
rpool/ROOT            3.08G   888G    96K  /rpool/ROOT
rpool/ROOT/pve-1      3.08G   888G  3.08G  /
rpool/data              96K   888G    96K  /rpool/data
rpool/swap            7.44G   895G  1.04G  -
vmdata                32.7G   866G    96K  /vmdata
vmdata/vm-100-disk-1   860M   866G   860M  -
vmdata/vm-101-disk-1  14.3G   866G  14.3G  -
vmdata/vm-101-disk-2   735M   866G   735M  -
vmdata/vmbackup       14.6G   866G    64K  -




Thank you
 
thanks for the reply,
so host 2 i get this

Code: 
root@prometheus2:~# zfs list -t snapshot
no datasets available

and on host 1 i get this

Code: 
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:30:01  2.11M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_20:45:01  1.47M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:00:01  1.48M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:15:01  1.22M      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:19:01     0B      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:20:01     0B      -  13.4G  -
vmdata/vm-123-disk-1@rep_test2_2018-03-23_21:21:01     0B      -  13.4G  -
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back