[TUTORIAL] Creating 2 node Proxmox VE cluster with StorMagic software defined storage

[brucekornfeld]

yes, it is using iscsi, but there is LVM shared volume group on top of that, so that should not be a problem !?

https://support.stormagic.com/hc/en...-SvSAN-HA-cluster-Part-5-Storage-Provisioning

See other post: https://forum.proxmox.com/threads/c...c-software-defined-storage.142472/post-639567
Storage based snaps are not supported at this time. (see screenshot image that shows this).

 

[brucekornfeld]

So essentially, this is a blatant vendor advertisement of a commercial solution that is "easier than Ceph", yet has none of the features Ceph provides.

Based on my reading this is essentially an iscsi target with thick LVM on top of it. If so, you appear to have zero understanding of what the community needs from shared storage.

Also, Let the folks in marketing know that a 2-node cluster + a "lightweight witness" is a 3-node cluster. And remember, THIS IS A SUPPORT FORUM, NOT A SALES CHANNEL.

>If I need a machine to run a witness on, why not just run Ceph on it?

because you can't run ceph on a low cost, low power machine like raspberry pi.

>So essentially, this is a blatant vendor advertisement of a commercial solution that is "easier than Ceph", yet has none of the features Ceph provides.

it think this is not blatant, because svsan provides some cool realtime iscsi mirroring solution for more then ten years.

i have used it in the past with vmware, and it rulez.

this is something what proxmox or opensource community won't have available, so it's an enrichment for everyone to know that this is now perhaps an option for being used with proxmox.

ok, ceph is integrated, but that's a completely different and complex beast with very high demand for hardware - and it's short-sighed to assume, that there or no other solutions which have their right to exist.

so, why being offensive? please don't.

i'd like to invite @brucek (Bruce Kornfeld?) to tell us about SvSAN plans on officially supporting proxmox.

So essentially, this is a blatant vendor advertisement of a commercial solution that is "easier than Ceph", yet has none of the features Ceph provides.

Based on my reading this is essentially an iscsi target with thick LVM on top of it. If so, you appear to have zero understanding of what the community needs from shared storage.

Also, Let the folks in marketing know that a 2-node cluster + a "lightweight witness" is a 3-node cluster. And remember, THIS IS A SUPPORT FORUM, NOT A SALES CHANNEL.

Thanks @RolandK for helping me explain. I am just sharing an alternative approach to ceph. Perhaps the disconnect here is that SvSAN was designed for small environments (edge/smb/robo) with limited budget and small hardware requirements (1vCPU, 1GB memory, 1 disk drive of any type). ceph requires a lot more than that. And yes - we do require 3 nodes (for proper quorum management), but our 3rd node doesn't have to be a full server - it is a small service that can run on anything (even raspberry pi if you want) and a single witness supports up to 1,000 clusters, only requires 9Kb/sec network connection and can withstand up to 3000ms latency. It was designed for edge use cases.

 

[brucekornfeld]

The problem I have is that this post is entirely misleading. It's not a solution to a problem. If I need a machine to run a witness on, why not just run Ceph on it? It would be ridiculous to jump through these hoops and have less functionality.
Besides, the best practice for Proxmox's high availability is three nodes. Why would I run 2? And will the Proxmox folks support this? Why am I running yet another cluster in a cluster?
I'm just saying this is a bad idea that's poorly thought out.

The difference is that the SvSAN witness is not a full node, each witness supports up to 1,000 clusters with minimal networking requirements (9Kb/sec). To answer your question on why you'd want to run 2 nodes is cost or space. The environments that are using SvSAN are typically edge and smb where a 2 server solution is preferable to 3 (airports, retail stores, oil rigs, manufacturing plants, wind farms, schools). If you need 3 nodes for processing power and storage - that is great and ceph is probably a perfect solution. And you wouldn't be running a cluster in a cluster. Glad to expand on that if desired.

 

[brucekornfeld]

>If I need a machine to run a witness on, why not just run Ceph on it?

because you can't run ceph on a low cost, low power machine like raspberry pi.

>So essentially, this is a blatant vendor advertisement of a commercial solution that is "easier than Ceph", yet has none of the features Ceph provides.

it think this is not blatant, because svsan provides some cool realtime iscsi mirroring solution for more then ten years.

i have used it in the past with vmware, and it rulez.

this is something what proxmox or opensource community won't have available, so it's an enrichment for everyone to know that this is now perhaps an option for being used with proxmox.

ok, ceph is integrated, but that's a completely different and complex beast with very high demand for hardware - and it's short-sighed to assume, that there or no other solutions which have their right to exist.

so, why being offensive? please don't.

i'd like to invite @brucek (Bruce Kornfeld?) to tell us about SvSAN plans on officially supporting proxmox.

Plans for supporting Proxmox: this is our first step. We published documents to help users that want to install SvSAN as the storage layer with Proxmox: https://support.stormagic.com/hc/en-gb/sections/14959498948381-SvSAN-Proxmox-7 We (StorMagic) will support users that wish to deploy SvSAN and to use it in production in a proxmox environment. Next up for us is to leverage the proxmox published APIs and plug-ins to make the deployment process easier.

 

[brucekornfeld]

@RolandK agreed... my tone is one of frustration. Apologies!
It just seems like pushing a square peg into a round hole. Ceph is integrated, supported by PVE, and free. And yes, if you really must, you can run it on a Raspberry Pi, NUC, or other... best of luck with that! IMHO, you are better off investing in the hardware rather than in software that attempts to work around investing in the hardware.
Happy to hear what @brucek has to say. But please, please, please... stop with the two-node solution masquerade. If I need a witness for high availability, I need a third "node."

Thanks for reply @pveuser113 I commented on other posts so this may be repetitive. I think the disconnect is that for some environments keeping the hardware costs to the absolute minimum is paramount. And our witness is a 1:1,000 design so that users with a large # of sites can deploy one witness for 1,000 sites with very small networking requirements (9Kb/sec, 3000 ms latency). Yes - the witness is a third node - not arguing that at all. Our largest customers has over 6,000 sites running 2 nodes each and they run 7 witnesses in their corporate datacenters (a single VM each with very little hardware allocated to it). The wtiness is only monitoring heartbeat and its not in the data path like most other witness / 3 node solutions out there.

 

[brucekornfeld]

> If I need a witness for high availability, I need a third "node."

but a full blown 3rd node is something you don't want to have in SMB/ROBO environemnt.

in the past, with SvSan you could even have the witness off-site, iirc.

so, you could have a full pve installation on some tiny, old and low-power hardware, same goes for svsan. that saves you a lot of money and energy waste. i don't have seen ceph with 2-node + witness/quorum node. so it's a different solution for different demand.

besides that, for quorum/whitness you typically need low-latency. for svsan, this is different. remote witness across wan still seems possible:

https://stormagic.com/svsan/features/witness/

correct. the witness can run anywhere - corp datacenter, cloud, windows laptop, raspberry pi, etc...etc...), 9Kb/sec network connection

 

[RolandK]

@brucekornfeld , did stormagic ever think on providing some free svsan license small virtual storage amount (i guess <<500gb and usage limitation for non prod is ok) for hobbyists and tech enthusiasts ? ( like leftand did for example https://community.hpe.com/t5/around...toreonce-vsa-1tb-data-protection/ba-p/6944232 )

furthermore, i have to admit that 15 days of eval period is a rather short timeframe ( i rally think it's awful short) . that scares me away personally.

if stormagic wants to delight and inspire people for your product and want to especially place it in proxmox orbit/territory, i guess some free but limited license would be helpful to get people on the boat , by attracting them via getting experience with it and using it.

i know at least 4 persons in my private orbit who run a proxmox cluster privately and non production , and i am running two. i search for quite a while for a ceph alternative to provide ha storage to such clusters, to get some experience with that and later on recommend for work.

furthermore, i know a lot who currently leave the sinking vmware boat , not least because of esxi free being revoked.

so, when vmware doing completely wrong (and they do, i have rarely seen such stupid and arrogant action) - stormagic could do something better/right. and get profit from that.

 

[brucekornfeld]

@brucekornfeld , did stormagic ever think on providing some free svsan license small virtual storage amount (i guess <<500gb and usage limitation for non prod is ok) for hobbyists and tech enthusiasts ? ( like leftand did for example https://community.hpe.com/t5/around...toreonce-vsa-1tb-data-protection/ba-p/6944232 )

furthermore, i have to admit that 15 days of eval period is a rather short timeframe ( i rally think it's awful short) . that scares me away personally.

if stormagic wants to delight and inspire people for your product and want to especially place it in proxmox orbit/territory, i guess some free but limited license would be helpful to get people on the boat , by attracting them via getting experience with it and using it.

i know at least 4 persons in my private orbit who run a proxmox cluster privately and non production , and i am running two. i search for quite a while for a ceph alternative to provide ha storage to such clusters, to get some experience with that and later on recommend for work.

furthermore, i know a lot who currently leave the sinking vmware boat , not least because of esxi free being revoked.

so, when vmware doing completely wrong (and they do, i have rarely seen such stupid and arrogant action) - stormagic could do something better/right. and get profit from that.

Great idea! Thanks for the feedback. I've already starting pursuing this internally. Stay tuned - I'll update this thread once I have a firm answer.

 

[dlasher]

>If I need a machine to run a witness on, why not just run Ceph on it?

because you can't run ceph on a low cost, low power machine like raspberry pi.

Actually you can. Healthy amount of cores, ram, and diskIO on the current generation PI/ARM/etc boxes.

 

[alexskysilk]

Actually you can. Healthy amount of cores, ram, and diskIO on the current generation PI/ARM/etc boxes.

you "can" isnt a particularly useful standard.

without proper networking or OSD bandwidth its not really useful for anything except for the lulz...