Advise on ceph additions: RGW, NFS, iSCSI

[godfather007]

Hi,

Does someone have a few pointers about this setup?

I want to install a ha object store and eventually some NFS volumes, using our ceph-installation, to use for our internal k8s cluster.

Is it better to get this virtualized to avoid upgrade issues on the physical hosts?
Does ceph-dashboard assist me in creating those using ceph-cli under the hood?

If i virtualize it, should i use a debian/ubuntu or pve installation?


Thanks for any help/info/insight,

Martijn

 

[spirit]

if you want to do pvc in your k8s cluster, you can do native rbd. (don't do nfs or iscsi).
you can use the rook operator in your k8s, and use the external ceph cluster from proxmox

https://rook.io/docs/rook/latest/CRDs/Cluster/external-cluster/



for radosgw for s3 object, I don't think you can use ceph-dashboard, because it can deploy only with cephadm && containerd, and proxmox install ceph with packages. (so cli-only)

 

[Natmac21007]

I'm looking for pointers on getting iSCSI to work with Ceph if someone has some experience. I have managed to get one gw to work but getting another saying unknown state when I try to add it as a portal in Ceph dashboard and I can't find any good info on doing it in the cli.

 

[godfather007]

if you want to do pvc in your k8s cluster, you can do native rbd. (don't do nfs or iscsi).
you can use the rook operator in your k8s, and use the external ceph cluster from proxmox

https://rook.io/docs/rook/latest/CRDs/Cluster/external-cluster/



for radosgw for s3 object, I don't think you can use ceph-dashboard, because it can deploy only with cephadm && containerd, and proxmox install ceph with packages. (so cli-only)

No NFS, iSCSI... alright... i'm still old-fashion i guess...
By native RBD you mean create RBD volumes using cephadm-cli and mount them through a client on various k8s VM's?

In that case i can also create RGW-s3 configuration manually on 2 physical hosts? Or you think it would be wise to make seperate VM's for those?

 

[jsterr]

@aaron any plans to switch your ceph deployment to cephadm? (as everything future-developed will be cephadm only) ceph dashboard in reef will have very nice additions you might also can use on cli base?

 

[godfather007]

Can pveceph also be used for creating swift/s3 pools?