Expose Ceph RBD to VMs for Kubernetes

thomas_l

New Member
Nov 5, 2018
5
1
3
31
Hello,

I'd like to use Ceph embedded inside Proxmox for a Kubernetes cluster running inside VMs. This means exposing Ceph RBD endpoint to VMs.

Two questions : is it possible ? is it supported ?

I would not use Ceph for anything other than persistent storage for Kubernetes. I have a Proxmox 5.4 cluster (ZFS storage).

Thank you !
 

spirit

Famous Member
Apr 2, 2010
4,517
332
103
www.odiso.com
Hi.

Yes, it should work. (you just need network access from your vm ip to ceph monitor/osd.)

Don't know how you'll implement support of rbd inside kubernetes, but the new rook implementation don't support yet external ceph cluster (not in kubernetes). But I known that rancher for example, have a rbd plugin to do it.
 

spirit

Famous Member
Apr 2, 2010
4,517
332
103
www.odiso.com
One remaining question : is it a supported way to use Ceph in Proxmox ? =)
Unsupported setup is a good way to have a problem with an upgrade.

yes, no problem.
Just be carefull when you upgrade from major proxmox (5->6 for example), as generally, the ceph version supported increase too.
(should be nautilus for the next major upgrade).
Maybe create a dedicated pool if one day you want to manage vms on ceph too.
(avoid to give permissions at kubernetes on the same pool than vm, for security purpose)
 

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
4,617
430
88
Two questions : is it possible ? is it supported ?
One remaining question : is it a supported way to use Ceph in Proxmox ? =)
If you ask with these questions, if Proxmox gives paid support for a Kubernetes + Ceph RBD setup? Then the answer is no. But from a technical standpoint, we build all packages from upstream, so there should be nothing hindering.
 

thomas_l

New Member
Nov 5, 2018
5
1
3
31
Thank you both for your answers !

If you ask with these questions, if Proxmox gives paid support for a Kubernetes + Ceph RBD setup? Then the answer is no. But from a technical standpoint, we build all packages from upstream, so there should be nothing hindering.
The question is not only about Kubernetes (even if it's what I plan to use), but more global; If I consume Proxmox Ceph RBD from inside VMs to create disks and mount them automatically, will it be a problem for support ?

If I understand correctly what you said, it's possible and supported, but of course you won't support what I do inside VMs (no support for Kubernetes obviously).

For now it's a homelab, but at some point, if it works correctly, I could decide to use it professionally, then : subscription.
 
  • Like
Reactions: Alwin

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
4,617
430
88
If I understand correctly what you said, it's possible and supported, but of course you won't support what I do inside VMs (no support for Kubernetes obviously).
Proxmox VE as client or Ceph cluster is under paid support.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!