Am I able to run Kubernetes clusters on Proxmox cluster?

[zmlman]

I was deploying / testing K8S cluster on my vmware cluster with an evaluation license, but I could not deploying it in our production environment, because the license for our production environment do not support DRS and resource pool feature which is only available in enterprise license.

So looking into the alternatives to vmware, I have few questions blow

1. does Proxmox provide the similar feature to vmware DRS which would allow us to run Kubernetes clusters?
2. does Proxmox support AWS EKS Anywhere local installation?
3. does Proxmox support SAN storage over fiber channel through fiber switch?
4. does Proxmox support local USB/SD Card installation?
5. is there a limitation for number of nodes that can be joined into a Proxmox cluster?

Thanks

 

[bbgeek17]

1. does Proxmox provide the similar feature to vmware DRS which would allow us to run Kubernetes clusters?

there is no built-in equivalent to DRS in Proxmox. There are some community written scripts. However, they are not functionally equivalent to DRS. If what you are installing is programmatically dependent on DRS then only VMware provides such API capability.

2. does Proxmox support AWS EKS Anywhere local installation?

AWS EKS A requires Docker or VMware. You can certainly install a few Ubuntu VMs in Proxmox and run Docker on them. This, technically, has nothing to do with Hypervisor (PVE).

3. does Proxmox support SAN storage over fiber channel through fiber switch?

The answer depends on what your definition of "support" is. Proxmox runs on Debian OS, you can certainly connect FC SAN to Debian. All the standard functionality at OS level is there as expected (multipath, etc). Its what you do with the storage at application layer is where you may have some challenges. Specifically - there is no builtin VMFS equivalent. It may or may not be an issue for you.

4. does Proxmox support local USB/SD Card installation?

Its my understanding that Proxmox is quite write-intensive and can wear out such devices quickly. Nothing stops you from trying.

5. is there a limitation for number of nodes that can be joined into a Proxmox cluster?

Not technically. There are quite large clusters out there.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[zmlman]

Thank you for your reply. @bbgeek17

1. What would you recommend if we want to create a production K8S cluster with HA features based on Proxmox?
2. Is there any limitation we should be aware of before we start building it?
3. Do you have any official docs for the instructions?
4. Do you know if there is any big company which is using Proxmox as their production environment?
5. Is there any compatibility issue with HP servers?

 

[bbgeek17]

1. What would you recommend if we want to create a production K8S cluster with HA features based on Proxmox?

The main thing that would be missing is an out-of-the-box automatic VM deployment control. You will likely have to build and integrate your own ansible/etc workflow. This is compared to Openstack and VMware integration that many vendors provide for you. Once you have VMs with your OS of choice deployed - installing and managing K8s should be no different than any other hypervisor.

2. Is there any limitation we should be aware of before we start building it?

You will need to make a list of required features and go through it. One thing that comes to mind - pay particular attention to shared storage integration if you need one.

3. Do you have any official docs for the instructions?

Not that I am aware of. You should reach out to official PVE support to inquire.

4. Do you know if there is any big company which is using Proxmox as their production environment?

You should reach out to official PVE support to inquire. You do plan to buy a license for this production(?) deployment?

5. Is there any compatibility issue with HP servers?

PVE is built on Debian, so in theory "anything that is supported with Debian, should be supported with PVE". In practice PVE maintains their own stable Kernel (as they should), so some targeted one-offs may not be available right away.
However, that is solved by working with PVE Support as a paying customer.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox