Storage and Networking suggestions in new Proxmox setup

[Sivakumar]

Hello Team,

Very Good Morning !
We are planning for 10 Host Proxmox setup. On top of that we are planning for around 150 VM's

Request you to comment on below 2 topics.

1. Will it be ok, if we use the server local disk for all VM's ? ( each Proxmox host or server will have own disks ) and not SAN or SAS storage.
2. Is there any problem in using 1 Gig port Network Switch for Proxmox environment ?

Kindly note that, all Proxmox Host's are connected within the LAN, I meant, within one room itself and that too only 2 switches.

Will you suggest us to use the network based storage or local disk ( inbuilt with server).

Appreciating your suggestions.

Many thanks in advance.


Regards
Sivakumar

 

[ozdjh]

Hi

Using ceph as your vm storage would be a good approach if you wanted to use drives in the pve nodes themselves. But you'd need to use 10GbE networking at least, and probably 2 different networks for ceph public and cluster traffic. Ceph performs well and provides you with good data redundancy but from our testing you must provide it with plenty of network bandwidth.


David
...

 

[Dominic]

Hello!

You can create a so called hyper-converged setup with Proxmox VE. There is a chapter about this topic in our documentation which should be very interesting for you I think. It initially states the various advantages of such a solution. Basically all the local disks of the hosts act as one storage in such a setup.

As @ozdjh mentioned, Ceph can do this and is officially supported by us (= part of subscription agreement). He is also right about the 10GbE. We did a benchmark with Ceph and different network connections which can be downloaded from our website.

 

[ness1602]

We have similar configuration. 6 hosts, 1gbit network, and local disks. Everything is working okay ATM, so this is always the cheapest road.

 

[Sivakumar]

Hello!

You can create a so called hyper-converged setup with Proxmox VE. There is a chapter about this topic in our documentation which should be very interesting for you I think. It initially states the various advantages of such a solution. Basically all the local disks of the hosts act as one storage in such a setup.

As @ozdjh mentioned, Ceph can do this and is officially supported by us (= part of subscription agreement). He is also right about the 10GbE. We did a benchmark with Ceph and different network connections which can be downloaded from our website.

Thank you Dominic.

I am planning to cluster the nodes. will hyper-converged setup help ? Because storage won't be accessbile, when I bring down the specific host. each physical server will have HDD.

I am bit concerned. Because we are planning this setup for production environment.

 

[Sivakumar]

We have similar configuration. 6 hosts, 1gbit network, and local disks. Everything is working okay ATM, so this is always the cheapest road.

Thanks ness. you meant, will it be ok with 1gbit network port for 150 VM loads on 10 Proxmox host ?

How many VMs are running on your Proxmox environment, since you said it is working fine with 1gbit network port.

 

[ness1602]

This is our current status, but i just installed 6th node

 

[Dominic]

I am planning to cluster the nodes. will hyper-converged setup help ? Because storage won't be accessbile, when I bring down the specific host. each physical server will have HDD.

Creating a cluster Proxmox VE is the base for a hyper-converged setup, it works together nicely. Ceph automatically stripes and replicates the data across the cluster. A certain amount of OSDs (disks) can fail without loosing data.

In addition to our Ceph benchmark, the hardware recommendations of Ceph are about networking as well.