[TUTORIAL] Creating 2 node Proxmox VE cluster with StorMagic software defined storage

How do you handle VM-Snapshots, it looks like svSAN is using iSCSI which should result it not beeing able to do snapshots in pve webui?
 
So essentially, this is a blatant vendor advertisement of a commercial solution that is "easier than Ceph", yet has none of the features Ceph provides.

Based on my reading this is essentially an iscsi target with thick LVM on top of it. If so, you appear to have zero understanding of what the community needs from shared storage.

Also, Let the folks in marketing know that a 2-node cluster + a "lightweight witness" is a 3-node cluster. And remember, THIS IS A SUPPORT FORUM, NOT A SALES CHANNEL.
 
That's okay, bbgeek17 is also a vendor representative, tkreinn(or similar) is a vendor representative.Nothing wrong with them being on forum.
Yes, iscsi is a bit simpler than CEPH, and this is a 2-node on steroids(since you need witness),whereas ceph won't work with two nodes.
 
ness1602 said:
Yes, iscsi is a bit simpler than CEPH, and this is a 2-node on steroids(since you need witness),whereas ceph won't work with two nodes.
Click to expand...
The problem I have is that this post is entirely misleading. It's not a solution to a problem. If I need a machine to run a witness on, why not just run Ceph on it? It would be ridiculous to jump through these hoops and have less functionality.
Besides, the best practice for Proxmox's high availability is three nodes. Why would I run 2? And will the Proxmox folks support this? Why am I running yet another cluster in a cluster?
I'm just saying this is a bad idea that's poorly thought out.
 
>If I need a machine to run a witness on, why not just run Ceph on it?

because you can't run ceph on a low cost, low power machine like raspberry pi.

>So essentially, this is a blatant vendor advertisement of a commercial solution that is "easier than Ceph", yet has none of the features Ceph provides.

it think this is not blatant, because svsan provides some cool realtime iscsi mirroring solution for more then ten years.

i have used it in the past with vmware, and it rulez.

this is something what proxmox or opensource community won't have available, so it's an enrichment for everyone to know that this is now perhaps an option for being used with proxmox.

ok, ceph is integrated, but that's a completely different and complex beast with very high demand for hardware - and it's short-sighed to assume, that there or no other solutions which have their right to exist.

so, why being offensive? please don't.

i'd like to invite @brucek (Bruce Kornfeld?) to tell us about SvSAN plans on officially supporting proxmox.
 
@RolandK agreed... my tone is one of frustration. Apologies!
It just seems like pushing a square peg into a round hole. Ceph is integrated, supported by PVE, and free. And yes, if you really must, you can run it on a Raspberry Pi, NUC, or other... best of luck with that! IMHO, you are better off investing in the hardware rather than in software that attempts to work around investing in the hardware.
Happy to hear what @brucek has to say. But please, please, please... stop with the two-node solution masquerade. If I need a witness for high availability, I need a third "node."
 
> If I need a witness for high availability, I need a third "node."

but a full blown 3rd node is something you don't want to have in SMB/ROBO environemnt.

in the past, with SvSan you could even have the witness off-site, iirc.

so, you could have a full pve installation on some tiny, old and low-power hardware, same goes for svsan. that saves you a lot of money and energy waste. i don't have seen ceph with 2-node + witness/quorum node. so it's a different solution for different demand.

besides that, for quorum/whitness you typically need low-latency. for svsan, this is different. remote witness across wan still seems possible:

https://stormagic.com/svsan/features/witness/
 
Last edited:
pveuser113 said:
If I need a machine to run a witness on, why not just run Ceph on it? It would be ridiculous to jump through these hoops and have less functionality.
Click to expand...
you would need the third node to have a pve cluster anyway- may as well use the pve witness for the storage too.

-edit-

In truth, I have often wondered why no such option exists for pve. Microsoft has had simple storage clustering using a on-disk quorum for many years- its a pretty primitive and effective solution.
 
Last edited:
>In truth, I have often wondered why no such option exists for pve

but it does?

besides qdevice (which does not integrate well into proxmox, as you won't notice failure easily fore example), you can have full blown pve on any old small x86 box/thin client out there for well under 20 bucks and <15W electrical power, and that's totally fine as a 3rd node, afaik. and it won't even need virtualization support in cpu, as you won't run any VM there.

https://forum.proxmox.com/threads/d...oday-try-ressource-constrained-device.117228/
 
Last edited:
I was under the impression that we were discussing clustered storage. to my knowledge, the only solution that was ever available as a two node cluster in pve was drbd, but it was dropped for either being unreliable or going commercial (I dont remember now.) the point is that such a solution can and should be available as it is technically simple.
 
>I think what @alexskysilk is referring to is a small physical disk, which is not the same as qdevce technology wise.

that small phyiscal disk needs to be "shared" between the two nodes, and with that requirement, it's not "small" anymore, as i need either NAS or SAN attached disk. that's the same or even higher level of cost then some old thin client.

i had run ibm svc at my former employee, do you want to know what it needed to have as a quorum?

a third san / fc attached storage in a different room/building. iirc, that was some IBM DS3400 and it was expensive, even with 2 disks....
 
alexskysilk said:
I was under the impression that we were discussing clustered storage. to my knowledge, the only solution that was ever available as a two node cluster in pve was drbd, but it was dropped for either being unreliable or going commercial (I dont remember now.) the point is that such a solution can and should be available as it is technically simple.
Click to expand...
For drbd,there was a overnight license change, so proxmox dropped them.
 
RolandK said:
yes, it is using iscsi, but there is LVM shared volume group on top of that, so that should not be a problem !?
Click to expand...

Thick LVM does not support snapshots [1] and LVM thin can't be used as shared storage. Haven't read the docs for svsan, do don't know which one uses and how.

AFAIK svsan comes from VMWare/Hyper-V world, where no ceph is (was?) possible. To me seems fair that they just propose their option as "working with ProxmoxVE". Maybe someone is migrating from other hypervisor to Proxmox and just want to reuse their svsan just because they are used to it. It's admin responsibility to evaluate every option and use which ever seems best for the use case.

[1] https://pve.proxmox.com/wiki/Storage#_storage_types
 
  • Like
Reactions: RolandK
alexskysilk said:
see windows failover clustering.
Click to expand...
That requires some shared storage (SAN/NAS) external to the cluster and acts just as a cluster quorum witness.

alexskysilk said:
I was under the impression that we were discussing clustered storage. to my knowledge, the only solution that was ever available as a two node cluster in pve was drbd, but it was dropped for either being unreliable or going commercial (I dont remember now.) the point is that such a solution can and should be available as it is technically simple.
Click to expand...
DRBD wasn't that nice. To be really reliable you needed to make it full sync (i.e. ack the write when written on both local and remote disk). Then your other node fails and you VMs fail because they can't write in the remote disk and renders the cluster useless. If you don't work synchronously (i.e. write to remote when you see fit), there's no guarantee that your data will be writen in the remote node and you may lose data.

There's even more situations were terrible things may happen... like you get a powerloss, both nodes down. For some reason the second node powers up but main node doesn't. You clusters starts VMs, apps make changes on disk. Another power loss. Then first node powers up but second doesn't... first node knows nothing about the changes made on second node. VMs start, new changes on disk, second node comes up and can't sync due to conflicts.

In fact, I see no real way to make 2 replicas work, as you are trading cost for data reliability.
 
  • Like
Reactions: pvps1 and pveuser113
jsterr said:
How do you handle VM-Snapshots, it looks like svSAN is using iSCSI which should result it not beeing able to do snapshots in pve webui?
Click to expand...
Correct, SvSAN does not do storage based snapshots (on the future roadmap to be prioritized with user feedback). We present up as block iSCSI and you are correct that pve snaps would not work with the current documented architecture. There are other approaches to the document that could introduce shared file systems, however may also require more than 2x nodes.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back