Proxmox 6.x + Ceph + CephFS

[shat]

I am currently evaluating Proxmox in a cluster environment and have intention to expand it to 7 storage nodes and 7 compute nodes to harness the storage provided by ceph. I have spent the last few weeks spending my effort formatting the machines and reinstalling everytime I make a ceph configuration mistake by which the guide and documentation says to reinstall as the best solution....

YAY

I hopefully have got to a point where I won't be formatting these machines again. However, I have run into a question related to CephFS.

Question

I created a CephFS filesystem named vm-storage and then ultimately removed it by shutting down the metadata servers; and removing it per this document discovered after surfing forum for others in situation.

https://pve.proxmox.com/pve-docs/chapter-pveceph.html#_destroy_cephfs

Follow-up Question

When using ceph distributed storage with Proxmox, should I be creating a pool and adding that as storage versus using cephfs as a whole?

At this point the vm-storage cephfs pool/path etc are gone but I still have . the storage option showing up in my tree (under datacenter in gui) and not sure how to get it to go disappear.

Suggestions?

* this cluster has no data I care about, yet *

 

[shat]

I feel absolutely idiotic! I didn't see the "Remove" section in Storage under DC.

If you have this issue, just goto the dc at top => storage => click the entry, remove.

Primary Question Answered! #solved

>> Follow-up Question

When using ceph distributed storage with Proxmox, should I be creating a pool and adding that as storage versus using cephfs as a whole?

 

[Stoiko Ivanov]

short answer: create a pool and use that as rbd storage.

cephfs is another layer on top of ceph and thus costs additional performance, without any gain for virtual machine disks
(one of the main initial reasons for adding cephfs was to be able to store backups and templates in ceph)

Hope this helps!