About Installation

[teknoadmin]

Hello!

I'm new at Proxmox. I used to VMware and I want to use Proxmox. We have 2.000-3.000 VM on our infra. What do you suggest about installation and daily usage in Proxmox? What should I use to get a permanent and healthy Proxmox environment?

Thank You!

 

[bbgeek17]

Hi @teknoadmin , welcome to the community. Sounds like impressive infrastructure. Deploying Proxmox is not unlike dealing with ESX, or any other hypervisor. There are 3 main pillars - Compute, Storage and Network. As long as you plan for day one and eventual growth, you should be ok.

Proxmox uses Debian with Ubuntu kernel as an underlying OS, so there is no official HCL beyond what is supported in Linux (almost everything). However, some things work better than others, and some are completely broken due to hardware vendor neglect.

For Compute - use good enterprise servers, preferably recent generation rather than from previous decade. Homogeneity is strongly recommended to facilitate problem free VM movement across servers. Single package CPU with high core count would give you best price/performance. Tally up your VMs and figure out how much Compute you should plan for.

For Network - stick to well known vendors, ie Mellanox. Plan for appropriate redundancy at each level, LACP is a popular option. Your choice of storage may also influence the number of ports/network/vlans you will want to have. Finally, 25G is the new 10G, get yourself 25G+. Based just on number of VMs - go with 100G.

Storage - your main options are HyperConverged, SAN and NAS. A Proxmox equivalent of ESX Vsan is Ceph. They share many of the positives and negatives. Enterprise SAN from one of the big vendors is also an option, however there is no built-in/supported equivalent to VMFS. NAS, ie NFS is an option - its use and functions are similar to how things work in VMware.
At Blockbridge we provide enterprise SAN storage on commodity hardware. Unlike all other SAN vendors we integrate natively with Proxmox API and provide VVol like functionality for Proxmox, with all the enterprise storage functionality one might expect.

Many of your choices will be influenced by your Region and budget. You may even want to reach out to one of the official Proxmox partners for a consultation.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[teknoadmin]

Hi @teknoadmin , welcome to the community. Sounds like impressive infrastructure. Deploying Proxmox is not unlike dealing with ESX, or any other hypervisor. There are 3 main pillars - Compute, Storage and Network. As long as you plan for day one and eventual growth, you should be ok.

Proxmox uses Debian with Ubuntu kernel as an underlying OS, so there is no official HCL beyond what is supported in Linux (almost everything). However, some things work better than others, and some are completely broken due to hardware vendor neglect.

For Compute - use good enterprise servers, preferably recent generation rather than from previous decade. Homogeneity is strongly recommended to facilitate problem free VM movement across servers. Single package CPU with high core count would give you best price/performance. Tally up your VMs and figure out how much Compute you should plan for.

For Network - stick to well known vendors, ie Mellanox. Plan for appropriate redundancy at each level, LACP is a popular option. Your choice of storage may also influence the number of ports/network/vlans you will want to have. Finally, 25G is the new 10G, get yourself 25G+. Based just on number of VMs - go with 100G.

Storage - your main options are HyperConverged, SAN and NAS. A Proxmox equivalent of ESX Vsan is Ceph. They share many of the positives and negatives. Enterprise SAN from one of the big vendors is also an option, however there is no built-in/supported equivalent to VMFS. NAS, ie NFS is an option - its use and functions are similar to how things work in VMware.
At Blockbridge we provide enterprise SAN storage on commodity hardware. Unlike all other SAN vendors we integrate natively with Proxmox API and provide VVol like functionality for Proxmox, with all the enterprise storage functionality one might expect.

Many of your choices will be influenced by your Region and budget. You may even want to reach out to one of the official Proxmox partners for a consultation.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

First of all, thank you for your kind reply. What do you suggest for installation? Which disk type should we use? ZFS or EXT4? Should we use RAID from our server's controller or Proxmox's? How can I find a Proxmox Partners by the way?

 

[LnxBil]

Which disk type should we use? ZFS or EXT4? Should we use RAID from our server's controller or Proxmox's?

Depends on if you want to use local storage or not (HA implies no local storage, you need dedicated or distributed shared storage). Easiest solution is ext4 on RAID controller and storage in the network (like in every other Hypervisor).

How can I find a Proxmox Partners by the way?

This official list.

 

[bbgeek17]

Adding on to @LnxBil , for boot storage we are fans of Dell BOSS solution, SuperMicro now has an equivalent that works quite well. I suspect other vendors might have one too.
For both Boot and Data storage you should use Enterprise (PLP) disks. Any major vendor would be fine, we are fans of Micron.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[teknoadmin]

Thank you very much again for all of you.

We are using local storage for now but we can build a shared storage system. Which one we should use you think? Also, how should I install proxmox? on a usb? on a ssd?

 

[bbgeek17]

We are using local storage for now but we can build a shared storage system. Which one we should use you think?

Without any indication about your requirements - its impossible to answer this question. Its also unclear if by local you mean HyperConverged/Distributed, ie Ceph, or truly local isolated disks.

Also, how should I install proxmox? on a usb? on a ssd?

Proxmox is a much more disk intensive system than ESXi. Its not recommended to install on USB if your desire is :

permanent and healthy Proxmox environment

You should install on enterprise grade mirrored NVMe.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Maximiliano]

First of all, thank you for your kind reply. What do you suggest for installation? Which disk type should we use? ZFS or EXT4? Should we use RAID from our server's controller or Proxmox's? How can I find a Proxmox Partners by the way?

Hello,

You can find our partners at https://proxmox.com/en/partners/explore.

Regarding the installation of Proxmox VE itself, a possible setup would be to have two SSDs (with power loss protection) having some kind of redundancy, e.g. in a ZFS mirror or hardware raid. The drives don't need that much capacity, 256GB should be enough. It is hard to say which one is best for you without full knowledge of your setup and requirements.

 

[teknoadmin]

Hi All! I will clarify my setup and reqs in a week, then I will send them to thread. Thank you!