Snapshots on NFS or iSCSI

[decibel83]

Hi,
I have a shared storage based on a NetApp system which exports through NFS or iSCSI.
I am not able to make snapshot backups for LXC containers on NFS storage, so I am looking for something better.
I read on https://pve.proxmox.com/wiki/Storage that even iSCSI does not support snapshot, but ZFS over iSCSI do.
But I am little bit confused, because I cannot understand if I need some special hardware to run it, or if I can setup it on with my existent NetApp storage.
Could you help me to understand better, please?
Thanks!

 

[pabernethy]

You don't need special hardware. You need software to support them. In this case the file system. Usually, when software writes to a file the state before a write is replaced by the state after the write. The previous state is lost. With snapshots the state of a volume is saved and when a file is written on the write is directed elsewhere, along with the information where this data belongs. So the new state is a compound of the snapshot (state before write) and the new data. Together they give the state after the write.
As you can guess, this becomes complicated very quickly as you have to track a lot of information. While it is possible to write software to do all of this on top of any file system, one would end up repeating many things the file system already does, so this task is usually integrated in the file system. ZFS supports snapshots, while iSCSI is not a file system, but a protocol exposing block devices via IP.
So saying iSCSI doesn't support snapshots is true for the same reason that any disk (no matter if it's attached via IDE, SATA, SCSI or something else) does not support snapshots. It doesn't do much on its own. It supports being given a bunch of bytes with an address to write them to.
Of the storage solutions Proxmox offers the ones supporting snapshots (from the host's point of view) are ZFS, LVM Thin, Ceph, Sheepdog and ZFS over iSCSI, as well as any file-based storage if QCOW2 images are used.

 

[hec]

Why not using NetApp Snapshots?

I think there is nothing like which works so great than the SnapShots on a NetApp in special you can have 255 SnapShots per volume.

If you need to restore you can do a FlexClone and mount the clone and then move the vm to the original nfs share.

It would be a nice feature that we could flush the caches before a SnapShot on the NetApp like we can do with vmware.
The we would have no data loss in case of a recovery from a SnapShot.

In more than 95% it is no problem but it would be a real nice feature.

For databases and exchange we can use SnapCenter so the guest drop caches is the only problem for a consistent SnapShot.

 

[fabian]

Why not using NetApp Snapshots?

I think there is nothing like which works so great than the SnapShots on a NetApp in special you can have 255 SnapShots per volume.

If you need to restore you can do a FlexClone and mount the clone and then move the vm to the original nfs share.

It would be a nice feature that we could flush the caches before a SnapShot on the NetApp like we can do with vmware.
The we would have no data loss in case of a recovery from a SnapShot.

In more than 95% it is no problem but it would be a real nice feature.

For databases and exchange we can use SnapCenter so the guest drop caches is the only problem for a consistent SnapShot.

netapp is a proprietary solution. you are free to write (or get someone to write) a storage plugin that uses features of your hardware - PVE even supports adding external storage plugins.

 

[hec]

Is it possible to drop the cashes with an api call?

If yes we could write a plugin this should be no problem. If not we would have normal backups which are not consistent. This is ok for the most servers but as i wrote above this would be a nice feature if we could do this.

With such an api call the support for snapshots of nearly every storage vendor would be possible.

 

[fabian]

no, dropping the system-wide caches is not possible via the API (and not necessary). we freeze the guest FS activity before making a snapshot for consistency, if needed.

 

[hec]

This would be also ok how can i do this?

I will talk monday or tuesday to our dev guys. Lets see if we can write a NetApp plugin.

Is it possible that you integrade this plugin and if yes which license do you need for the plugin?

 

[wolfgang]

May be this plugin will do what you need.

https://github.com/odiso/proxmox-pve-storage-netapp

 

[hec]

I know this plugin but as i see this is very old and not using the NetApp API. I'm not sure if this plugin is working with Proxmox 5 but i will test it on Monday in our Lab.

Maybe we can use this as a base and rewrite the code to the NetApp API.

 

[Tmanok]

Of the storage solutions Proxmox offers the ones supporting snapshots (from the host's point of view) are ZFS, LVM Thin, Ceph, Sheepdog and ZFS over iSCSI, as well as any file-based storage if QCOW2 images are used.

Adding to this: File-based means mountpoint on a local filesystem (ext4 on single disk or HW RAID for example). QCOW2 specification is important because .raw disks (often converted from VMWare or HyperV), for both VMs and the default for LXCs, have different requirements for snapshots.

Are there future plans to improve LXC snapshot capabilities? Not every client that I work with can afford a CEPH solution and many have existing iSCSI and NFS SANs for their shared storage. Don't get me wrong, I am infatuated with CEPH, it is an amazing system, but many non-profits simply cannot afford such an enterprise solution.

Cheers, sorry to bump an ancient thread, it is the "latest" that is relevant to my question.


Tmanok