Expose Ceph RBD to VMs for Kubernetes

thomas_l

Member
Nov 5, 2018
5
1
8
35
Hello,

I'd like to use Ceph embedded inside Proxmox for a Kubernetes cluster running inside VMs. This means exposing Ceph RBD endpoint to VMs.

Two questions : is it possible ? is it supported ?

I would not use Ceph for anything other than persistent storage for Kubernetes. I have a Proxmox 5.4 cluster (ZFS storage).

Thank you !
 
Hi.

Yes, it should work. (you just need network access from your vm ip to ceph monitor/osd.)

Don't know how you'll implement support of rbd inside kubernetes, but the new rook implementation don't support yet external ceph cluster (not in kubernetes). But I known that rancher for example, have a rbd plugin to do it.
 
One remaining question : is it a supported way to use Ceph in Proxmox ? =)
Unsupported setup is a good way to have a problem with an upgrade.

yes, no problem.
Just be carefull when you upgrade from major proxmox (5->6 for example), as generally, the ceph version supported increase too.
(should be nautilus for the next major upgrade).
Maybe create a dedicated pool if one day you want to manage vms on ceph too.
(avoid to give permissions at kubernetes on the same pool than vm, for security purpose)
 
Two questions : is it possible ? is it supported ?
One remaining question : is it a supported way to use Ceph in Proxmox ? =)
If you ask with these questions, if Proxmox gives paid support for a Kubernetes + Ceph RBD setup? Then the answer is no. But from a technical standpoint, we build all packages from upstream, so there should be nothing hindering.
 
Thank you both for your answers !

If you ask with these questions, if Proxmox gives paid support for a Kubernetes + Ceph RBD setup? Then the answer is no. But from a technical standpoint, we build all packages from upstream, so there should be nothing hindering.
The question is not only about Kubernetes (even if it's what I plan to use), but more global; If I consume Proxmox Ceph RBD from inside VMs to create disks and mount them automatically, will it be a problem for support ?

If I understand correctly what you said, it's possible and supported, but of course you won't support what I do inside VMs (no support for Kubernetes obviously).

For now it's a homelab, but at some point, if it works correctly, I could decide to use it professionally, then : subscription.
 
  • Like
Reactions: Alwin
If I understand correctly what you said, it's possible and supported, but of course you won't support what I do inside VMs (no support for Kubernetes obviously).
Proxmox VE as client or Ceph cluster is under paid support.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!