Multipathing with SAS storage but still be able to snapshot

[RoxyProxy]

Hi there,
I currently have a Proxmox Cluster with 2 identical Hosts with SAS cards and a V3700 V2 Storage that has each storage-canister directly attached to both Hosts with SAS cables.

I was able to successfully configure multipathing and create an LVM on it, however, as the documentation says, I'm not able to take snapshots of VMs with LVM.

Sadly I wasn't able to find much about SAS Storage connection with Proxmox, most I found was about iscsi.

What ways are there to have the multipathed directly connected storage and still be able to take snapshots?

sorry, Im not very versed with storages and filesystem stuff and so on, neither with proxmox tbh but Im getting there xD

Any help would be greatly appreciated

 

[bbgeek17]

What ways are there to have the multipathed directly connected storage and still be able to take snapshots?

The primary way is to ditch LVM and implement a Cluster-aware filesystem (OCFS2 or GFS). Technically, you could place CAF on top of LVM as well.

Note that CaFs are not directly supported by PVE, nor do they have a GUI button to make it just work. There are many guides online that may walk you through. There have been several threads about this in the last 7 days. I'd recommend reviewing them.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[RoxyProxy]

Thanks for the quick answer!

I will take a look at this

 

[Falk R.]

Whether you use iSCSI, FC or SAS does not matter, they are just different transfer protocols for SCSI packets.

Why do you necessarily need snapshots? Most people only want it because they are used to it from VMware.
You can still perform snapshot backups even with LVM without snapshot functionality. If you have typically taken a snapshot before a software update, you can better make a backup. You can restore this if the update ruins everything, but usually only individual files or settings are lost. This can be restored much better from a backup than rolling back the snapshot every time.

 

[alexskysilk]

I was able to successfully configure multipathing and create an LVM on it, however, as the documentation says, I'm not able to take snapshots of VMs with LVM.

As @bbgeek17 said, thats because thats not an option. HOWEVER, it is possible to utilize the storage backend to accomplish this even if it will not be hosts aware. I'm not familiar with the Storwize productline but I have to imagine there is an API available for you to call and issue a hardware level snapshot. It would necessarily be a hack you would have to develop.

Note that CaFs are not directly supported by PVE, nor do they have a GUI button to make it just work. There are many guides online that may walk you through. There have been several threads about this in the last 7 days. I'd recommend reviewing them.

I'd add that the available options on Linux/Debian (OCFS/GFS2) are not well implemented and suffer various limitations and poor performance.

Why do you necessarily need snapshots? Most people only want it because they are used to it from VMware.

Snapshots are FAR easier and faster to commit/revert/mount then backups. Consequently, they are essential when you have a service level commitment. It has everything to do with scope; In OPs case, I tend to agree with you.

 

[waltar]

I was able to successfully configure multipathing and create an LVM on it, however, as the documentation says, I'm not able to take snapshots of VMs with LVM.

As a V3700 v2 is an external hw raid system you could even create a LVM-thin onto the multipath'ed volumes to take snapshots of VM's by pve or use the snapshot functions inside the storage system. A cluster aware filesystem with 2 nodes will make really no sense, just let one fail ...

 

[bbgeek17]

I currently have a Proxmox Cluster with 2 identical Hosts with SAS cards and a V3700 V2 Storage that has each storage-canister directly attached to both Hosts with SAS cables.

As a V3700 v2 is an external hw raid system you could even create a LVM-thin

This will lead to data corruption.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[waltar]

This will lead to data corruption.

Sure, this is why Dell, HP, Lenovo and others still sell hw raids and the customers still buy and use it as it's so bad in the last decades

 

[bbgeek17]

Sure, this is why Dell, HP, Lenovo and others still sell hw raids and the customers still buy and use it as it's so bad in the last decades

I may be missing your point, I apologize.

What I was saying is that using an external storage that is simultaneously connected to more than one host (for the same LUN) and then placing _thin_ LVM layer on that LUN will lead to data corruption.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[waltar]

Yes, that's right when using same time that volume ... but that's normal when doing ha-nas and in case of one node is failing the other take over the volume and filesystem for exporting to nas clients.
But anyway using lvm or lvm-thin on a volume under a filesystem (when using) is ever an additional i/o layer which can fail for unwritten blocks in a power outage ... but again still asuming pve users are prefering block storage instead of file storage.
By me I find file storage much easier and usage clearer to understand as block storage looks more complex in space usage and trimming etc.

 

[bbgeek17]

Yes, that's right when using same time that volume ... but that's normal when doing ha-nas and in case of one node is failing the other take over the volume and filesystem for exporting to nas clients.

This particular conversation is in reference to PVE, where multiple nodes in the cluster are accessing the LUN concurrently. It seems like you are thinking of a different product.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[alexskysilk]

Sure, this is why Dell, HP, Lenovo and others still sell hw raids and the customers still buy and use it as it's so bad in the last decades

those are hardware vendors. file system coherency is handled by the host operating systems. PVE has no capability to do so on thin provisioned LVM.

 

[Falk R.]

Sure, this is why Dell, HP, Lenovo and others still sell hw raids and the customers still buy and use it as it's so bad in the last decades

If you use a hypervisor that works file system centric, then such a storage is a great solution. For ESXi, HyperV and Xen it makes it easier to consume and set up.
Proxmox has just gone the block- centric way and can therefore achieve better performance, but does not have a cluster file system in the repository for monolithic storages.
Both are not bad for the respective use case and I have learned to love the Proxmox way. I achieve up to double SQL I/O performance with identical hardware and VM sizing, just because you can save the additional file system.