Using PROXMOX as a Provisioning Tool

[GrahamLeach]

(Please forgive me if this topic is well covered elsewhere, if it has been, please point me in the right direction?)

I am wondering if it is possible to use PROXMOX to explore the minimum system sizing required for different customer profiles?

I notice that a lot of the offerings (i.e. the "overwhelming majority") out there from hosting companies are infrastructure-focused.

I want to go after this problem from a different direction. Presumably, general classes of website customer have emerged over the decades.

Some of the website "classes" that come to mind are:

- Static site, HTML only
- CMS-driven site, script driven
- Web Application Framework Site, library driven
- eCommerce Site
- eGovernment Site
....etc...

Has a corresponding "best fit" profile for these different classes also emerged, due to market forces?

Q: Have well-understood customer classes emerged?
Q: Have well-understood hosting profiles emerged to match the well-understood customer classes?

I presume that other people would agree with me that these "two sides of the same coin" exist, and I would like to know what they know.

My immediate challenge is sizing the hosting for an eCommerce site with 100 well-described products and about 2 unique visits per minute.

The kind of information I seek is specific to the above line of inquiry, not "advice" that this kind of exercise is impossible to conduct due to the infinite variations, nuances and complexities of the problem space being examined. If that were true, nothing on earth would be "fit to purpose" - and yet we are surrounded by examples of them.

Specific to my line of questioning is, can I?

A) Use PROXMOX to simulate "weak" to "strong" server profiles (CPU/RAM/DISK/NET)
B) Install representative examples of "customer profiles" (I have about 50 examples to pick from already)
B) Use a separate system to execute stress tests against the sites hosted on the PROXMOX server, under controlled conditions (LAB network)
C) Discover/Uncover/Validate "advised" generic Server Profiles that match generic Customer Profiles

If PROXMOX has already been used to conduct this type of research, I would love to be exposed to the methodology, findings and conclusions.

N.B.: This is essentially a "regression to the mean" exercise.

 

[FrankList80]

This is an interesting topic. As you said, its usually Infra based perspectives taken here. I could imagine with PVE API interfaces you could do that research on your own utilizing extensive Ansible/Terraform scripts to ramp this up and test. A major exercise i would assume pre-provisioning all these Skelleton-VMs / Containers / LXC. P.S: I would assume with "Server Profiles" you mean Virtual VMs ... not Hardware.

---

​

[Virtualization Support for SME and NGOs | DLI Solutions ]

 

[GrahamLeach]

This is an interesting topic. As you said, its usually Infra based perspectives taken here. I could imagine with PVE API interfaces you could do that research on your own utilizing extensive Ansible/Terraform scripts to ramp this up and test. A major exercise i would assume pre-provisioning all these Skelleton-VMs / Containers / LXC. P.S: I would assume with "Server Profiles" you mean Virtual VMs ... not Hardware.

​

---

[Virtualization Support for SME and NGOs | DLI Solutions ]

Hello,

Yes, I am thinking of simulating progressively more powerful "server profiles" using PROXMOX as the simulator testbed.

What I am not seeing in the industry is a "rightsizing" service where vendors allow a site to "gear down" rather than "gear up".

Think of it like a "dutch auction" for server provisioning, instead of starting low and cranking things up, start high and "settle down".

I *could* go out and buy 1 example of everything (my city is awash in eWaste) but then I wouldn't need PROXMOX, would I? <joke>

 

[ghusson]

On your VM configuration you can limit vCPU number, RAM size, disks IOs and bandwidth and network bandwidth. So almost all resources allowed to a VM can be controlled. What can't be simulated and controlled is an infrastructure saturation by others VMs (this case can be encountered when you use no garanted VM servers at cloud provider for example), with a random pattern. So why wouldn't be able to make your project to Poxmox ? For automation, I personally would use CloudInit and Ansible.

 

[GrahamLeach]

On your VM configuration you can limit vCPU number, RAM size, disks IOs and bandwidth and network bandwidth. So almost all resources allowed to a VM can be controlled. What can't be simulated and controlled is an infrastructure saturation by others VMs (this case can be encountered when you use no garanted VM servers at cloud provider for example), with a random pattern. So why wouldn't be able to make your project to Poxmox ? For automation, I personally would use CloudInit and Ansible.

@ghusson - Thanks for your comment. Well understood.

Here's some additional information so you can help me figure out if PROXMOX is the right simulation/benchmarking tool for me:

(1) I have purchased a dedicated server so as to be able to perform these tests in a controlled environment.
(2) I have implemented a separate physical network so as to better control network utilization / saturation
(3) I have a number of machines that can be used to help simulate client demand

Based on your comments, it looks like I can use PROXMOX to simulate "hosts" of varying power instead of having to buy them. Great.

I want to perform TTF testing with respect to individual eCommerce websites to see where things start to break down in the LAB.

After that, the next challenge would be to install the website(s) on real-life providers and see where they break down in the FIELD.

Analysis of the difference between LAB and FIELD breakdowns will undoubtedly uncover some very interesting findings.

The parameters I want to measure in terms of TTF ("Test to Failure") are the classic resource bottlenecks (CPU/RAM/DISK/NET)

Having supplied this additional information, do you have anything to add that might accelerate me with this project?

Thanks!

 

[ghusson]

@ghusson - Thanks for your comment. Well understood.

Here's some additional information so you can help me figure out if PROXMOX is the right simulation/benchmarking tool for me:

(1) I have purchased a dedicated server so as to be able to perform these tests in a controlled environment.
(2) I have implemented a separate physical network so as to better control network utilization / saturation
(3) I have a number of machines that can be used to help simulate client demand

Based on your comments, it looks like I can use PROXMOX to simulate "hosts" of varying power instead of having to buy them. Great.

I want to perform TTF testing with respect to individual eCommerce websites to see where things start to break down in the LAB.

After that, the next challenge would be to install the website(s) on real-life providers and see where they break down in the FIELD.

Analysis of the difference between LAB and FIELD breakdowns will undoubtedly uncover some very interesting findings.

The parameters I want to measure in terms of TTF ("Test to Failure") are the classic resource bottlenecks (CPU/RAM/DISK/NET)

Having supplied this additional information, do you have anything to add that might accelerate me with this project?

Thanks!

I would say that your test platform will not correspond at 100% to the real-life providers platforms. I expect you to see big differences, like factor 2.
Be careful to cache levels that your test platform can enable (and that you shouldn't have in real-life providers platforms). Some documentation : https://www.baeldung.com/linux/file-system-caching
It can lead to huge differences in IOs. Idem for RAID card disk caching for writes.
In order to see resources usage, I would recommend Zabbix to configure custom probes, 5s delay between metrics points.
Interesting project, have fun ;-)