Advice on filesystem for fast snapshot and backup

K

ktecho

Active Member
Jun 6, 2016
49
0
26
44
Hi there,

I need to setup a pair of Proxmox servers. I think I'm not using full HA, because I don't know yet if I'm having shared storage. So by now I'll think that both servers will have independent Proxmox instance.

The problem I have with my current VMs is that they get stuck while doing the snapshot/backup, I think because of the RAM dump. Some of my VMs get stuck during 5 or 10 minutes. What I would like to do is having the possibility of doing live snapshots. That way I can do some of them during the day (not including RAM) so I have rollback points and I can make a nightly backup (including RAM). I'm assuming that if you want to save RAM state, the VM is going to freeze during that time, but I would like that time to be as small as possible. The VMs are not going to have a very high CPU load or anything.

I readed a lot of good stuff on ZFS, but in the forums people seems to be having problems at least if you don't have a lot of RAM, so maybe LVM-thin is a better choice.

What would you recommend me if I that are my main concerns?
 
Forgot to say that VMs are probably going to be LXC, but I could consider changing to KVM if it has any advantages. Thanks!
 
I don't know what would be the best, but I'm running my VM's on BTRFS and this filesystem is terrible. I have 4 disks in BTRFS RAID 10 and 1 disk for backups. The disk with the backups had to be rebalanced all the time because it ran out of space, while there is still enough free space. (more info about this behaviour: http://askubuntu.com/questions/464074/ubuntu-thinks-btrfs-disk-is-full-but-its-not ) I've migrated that disk back to ext4. On my RAID10 (they are brand new SSD disks) I had uncorrectable errors and 1 VM disk got corrupt. After moving my VM's to the backup disk, BTRFS could correct the errors on the RAID because there was no data on it any more. I moved back the VM's to the RAID and 1 week later I have again uncorrectable errors. For the moment I'm also looking for a good alternative...
 
I had the same problem some weeks ago and went with ZFS in the end. The only alternative (At least: IMHO) right now would be mdadm if it's just one machine.

Regards, Jonas
 
ZFS and LXC work very well. You just can create a snapshot (<1 sec operation) and send/receive it off-site and you have your backup. Easier than that is simply not possible. The only thing you need to take care of is to have a consistent backup, so e.g. for databases, do a log switch and sync before creating the snapshot.
 
LnxBil said:
ZFS and LXC work very well. You just can create a snapshot (<1 sec operation) and send/receive it off-site and you have your backup. Easier than that is simply not possible. The only thing you need to take care of is to have a consistent backup, so e.g. for databases, do a log switch and sync before creating the snapshot.
Click to expand...

What about the RAM stuff? Do you really need half of the system RAM dedicated to ZFS? I've read reports about machines rebooting with no reason maybe because of the low RAM. I'm a little scared about this...

Besides that, I would like to have complete backups (including RAM) because I don't know if you can trust that one of this ZFS snapshots will boot a newly created machine if someone has not synced ok, but I guess if you don't have very fast disks, dumping the RAM to disk is going to freeze the machine for minutes regardless of the filesystem.
 
Hi,

I am looking zfs too as a file system to get a fast way to recover.zfs snapshots are storage level backup so that is not application aware.But If you do snapshot send/receive every 15 min for example you will get a few snapshot to recover and if the latest one bring your vm in an inconsistent status you could restore for one less recent.
That depend if you could afford to lose 30 min or more of data or not.As a disaster recovery i think it is a way to go.
For dayly backup a think it's even a beter solution than vzdump cause you could recover backups more recent thant a daily backup vzdump.
But just to get more insurance you could do a nightly vzdump in addition.

The only negative point for zfs for me is that you can not do live migration of vm since it is not a shared storage.So if you have a cluster and your node is getting bad(that same node with zfs storing your vms disks)you can not just migrate the vm and reboot the node shutdown to get repaired...

Sincerly

ktecho said:
What about the RAM stuff? Do you really need half of the system RAM dedicated to ZFS? I've read reports about machines rebooting with no reason maybe because of the low RAM. I'm a little scared about this...

Besides that, I would like to have complete backups (including RAM) because I don't know if you can trust that one of this ZFS snapshots will boot a newly created machine if someone has not synced ok, but I guess if you don't have very fast disks, dumping the RAM to disk is going to freeze the machine for minutes regardless of the filesystem.
Click to expand...
 
Forget my comment on btrfs on raid10; seems I was running raid5 which is indeed unstable due to errors in calculating the parity bit. I've reformatted it to RAID10 now. This is considered stable.
 
Sincerely I did the tour of this question a lot of times : Which FS is the Best ?

While ext3 was populars, I was using ReiserFS.

I love BtrFS principle... to young, too unstable, ...
I used it few times as Backend FS with Docker and in OpenSUSE it comes by defaults.
XFS was my unique choice for a long times, because he it quick with tiny and huge files and super stable.

But since I switched to ZFS
- yes you need at least 4GB of RAM to make it comfortable but come on, you do virtualization or not ?

I reach performance with LXC & KVM I'd never did before, even with LVM. I means like 200-500% MB faster;
Ok of course I use a SSD's for zlog and zcache; but that's the way..

...
So to build my Proxmox from scratch I start with:
like they explain on github zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS

than make partition for different zpool even it's not in the Best Pratice of ZFS
...

So for backup I put them on USBDrive using, .. , ZFS ?

Nope; Because my backups goes offsite and I prefer security of insanity.
So they are crypted with LUKS and formatted with XFS
which I automount like they explains at howtoforge(dot)com / automatically-unlock-luks-encrypted-drives-with-a-keyfile
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back