Setup a small Cloud

[jihadbek]

Hi,
my technical background is not solid , so i will try my best to explain ...
Currently we have several live websites and we are hosting each of them on a dedicated server and everywhile we will need to upgrade the hardware based on the load , so we thought for a solution so the upgrade be much easier and to be on demand , for example some times there is a promotion which cause much more load on website , so we upgrade only during that time and return to normal after finish , this is only possible with cloud .
also sometimes a website need resources of several servers to handle the load.

the ideal is to setup and configure a cloud of several dedicated servers to host our websites and maybe selling locally to some clients as well .

1 - Does Proxmox support this ?
2 - my knowledge is not enough to make initial setup and config for it , does proxmox support cover this ?
3 - Any suggestions for my case considering that we have a relatively low budget.

 

[Vladimir Bulgaru]

Hey!

Regarding the hosting, this is a very specific use case and it depends on your stack and physical setup. Let me dive a bit deeper into some of the aspects:

1. You've said you're hosting each website on a dedicated server. I assume you mean VPS, otherwise it is interesting why you are looking into virtualisation. In case of VPSs you can change resource allocation, but most of the times it requires a shutdown of the VM (CPU and RAM upgrades).
2. You need to look into the bottleneck. Most of the times it's either CPU, RAM or IOPS. The bottleneck needs to be addressed first. In case of CPU and RAM you can over-provision CPU and RAM, so that the load is dynamically balanced. It's bad if all the VMs peak simultaneously, though. As for IOPS - get good SSDs (PCIe / NVMe).
3. Proxmox allows creating clusters with High Availability VMs. This allows the VMs to stay up even if one server is under huge load (it gets balanced by the other server or servers).
4. Maybe VMs is not the way to go. I'd suggest using containers, since they are more lightweight and easier to reload if needed.

Hope these suggestions help.

 

[jihadbek]

Hey!

Regarding the hosting, this is a very specific use case and it depends on your stack and physical setup. Let me dive a bit deeper into some of the aspects:

1. You've said you're hosting each website on a dedicated server. I assume you mean VPS, otherwise it is interesting why you are looking into virtualisation. In case of VPSs you can change resource allocation, but most of the times it requires a shutdown of the VM (CPU and RAM upgrades).
2. You need to look into the bottleneck. Most of the times it's either CPU, RAM or IOPS. The bottleneck needs to be addressed first. In case of CPU and RAM you can over-provision CPU and RAM, so that the load is dynamically balanced. It's bad if all the VMs peak simultaneously, though. As for IOPS - get good SSDs (PCIe / NVMe).
3. Proxmox allows creating clusters with High Availability VMs. This allows the VMs to stay up even if one server is under huge load (it gets balanced by the other server or servers).
4. Maybe VMs is not the way to go. I'd suggest using containers, since they are more lightweight and easier to reload if needed.

Hope these suggestions help.

Great Answer , Thanks

Regarding our websites , Each in a separate Dedicated because one is not enough to handle CPU Load sometimes.
my benefit of the cloud will be using 5 servers for example and then create three VMs for our three websites and arrange shared resorces of the 5 servers on these three , or just creating one VM with the maximum resources and host the three websites on it so the cloud platform would load-balance the load on resources between the 5 physical servers , and then we can expand by adding 6th server to the cloud .. 7th .. 8th etc..

so is this possible ? and can it be done under low budget ?

 

[guletz]

Regarding our websites , Each in a separate Dedicated because one is not enough to handle CPU Load sometimes.
my benefit of the cloud will be using 5 servers for example and then create three VMs for our three websites and arrange shared resorces of the 5 servers on these three , or just creating one VM with the maximum resources and host the three websites on it so the cloud platform would load-balance the load on resources between the 5 physical servers , and then we can expand by adding 6th server to the cloud .. 7th .. 8th etc..

Hi,

So, if I understand, you may need this:

Each of your webservers(3) are idententical, but you only want to split the load on all of them them(using 5 harware node)
- in this case you can create at least 1or 2 web-server one each hardware node(3 at minimum)
- on each hardware node you can use HAProxy/ucarp and do load-balancing AND fail-over for your clients
- so even with one hardware node (or even 4 from 5 )is down, your clients will be able to use your web-site
- even better, you could also make a script to get the existent load using only 3-4 hardware nodes, and on a warning limit, you can power one another hardware node ore maybe all, and when the load is less then the warrning limit, you can power down some hardware nodes(for less electricity consumption)

This ideea can work with PMX and it also scale, in the case that you will add more hardwate nodes in the future, or even if you want to eliminate nodes from your cluster.

Good luck!

 

[jihadbek]

Hi,

So, if I understand, you may need this:

Each of your webservers(3) are idententical, but you only want to split the load on all of them them(using 5 harware node)
- in this case you can create at least 1or 2 web-server one each hardware node(3 at minimum)
- on each hardware node you can use HAProxy/ucarp and do load-balancing AND fail-over for your clients
- so even with one hardware node (or even 4 from 5 )is down, your clients will be able to use your web-site
- even better, you could also make a script to get the existent load using only 3-4 hardware nodes, and on a warning limit, you can power one another hardware node ore maybe all, and when the load is less then the warrning limit, you can power down some hardware nodes(for less electricity consumption)

This ideea can work with PMX and it also scale, in the case that you will add more hardwate nodes in the future, or even if you want to eliminate nodes from your cluster.

Good luck!

Well, since i dont understand fully how proxmox works i thought with more simplified mind to find a solution for our case to make the upgrade/downgrade easier and more flexible compared to dedicated servers.
maybe it is not as simple as i thought , i thought that creating a cloud to virtually consolidate several dedicated server resources ( CPU Cores , RAM , Disk .. ) and then create what i want as a virtual machine with the specs i want and upgrade it or downgrade it on demand .

i had a simple calculation in my mind ( 5 server each one have 12 core cpu with give me a 60 core i can assign whatever i want .. and so with ram and disk space )

 

[guletz]

Hi,

Proxmox is not a private cloud system. But like I said you can get in the end the same result, but not as you imagine / think.

As someone from China said in the past: "is not important what is the color of the cat, but only if she can catch a mouse."

But if you have only a limited budget, I think you can not get a private cloud as you want(with dinamic allocation of resurces as your load is change).

Good luck

 

[jihadbek]

Hi,

Proxmox is not a private cloud system. But like I said you can get in the end the same result, but not as you imagine / think.

As someone from China said in the past: "is not important what is the color of the cat, but only if she can catch a mouse."

But if you have only a limited budget, I think you can not get a private cloud as you want(with dinamic allocation of resurces as your load is change).

Good luck

You are right ..
so now we get to the other question , how and who can help me setup such a good solution for hosting our websites and as well selling to other in small scale ..

 

[alexskysilk]

for example some times there is a promotion which cause much more load on website , so we upgrade only during that time and return to normal after finish , this is only possible with cloud .
also sometimes a website need resources of several servers to handle the load.

Your problem can be solved in a number of ways, but the most impactful would be to adapt your website to docker/kubernetes. what that will allow you is to scale your compute resources with load- and you can run it anywhere including on proxmox. This isnt easy and would require a substantial rewrite of your website, but its the correct way to accomplish what you're after- you can use your existing servers as primary and scale out to a third party vendor such as gcp.