I have been able to use the new CLI tooling in 7.2 to create EC pools but I can't figure out how to use the data pool for cephfs. Any help is appreciated.
Ceph EC pool for CephFS with new CLI tools?
- Thread starter jasonsansone
- Start date
Hi,
you could use the
you could use the
--add_storage 0 option for pveceph pool create when creating the pools and then use ceph fs new to create a CephFS on top of them. However, Ceph discourages usage of EC pools for CephFS.
Hi Fiona,
Thank you very much. That option worked really well.
Managed to create a new cephfs using the proxmox default tools and attached an EC pool to it and assigned it to a folder.
Now I am copying files to the EC pool and interestingly enough I'm not seeing any objects in the default data pool.
I'm wondering if that issue in ceph is still up to date ....
Thank you very much. That option worked really well.
Managed to create a new cephfs using the proxmox default tools and attached an EC pool to it and assigned it to a folder.
Now I am copying files to the EC pool and interestingly enough I'm not seeing any objects in the default data pool.
I'm wondering if that issue in ceph is still up to date ....
Code:
pveceph pool create cephfs_ec --erasure-coding k=2,m=1
ceph osd pool application enable cephfs_ec-data cephfs --yes-i-really-mean-it
ceph osd pool application disable cephfs_ec-data rbd --yes-i-really-mean-it
ceph fs add_data_pool cephfs cephfs_ec-data
mkdir /mnt/pve/cephfs/ec
setfattr -n ceph.dir.layout.pool -v cephfs_ec-data /mnt/pve/cephfs/ec
Code:
root@pvenew:~# ceph df
--- RAW STORAGE ---
CLASS SIZE AVAIL USED RAW USED %RAW USED
hdd 75 TiB 38 TiB 38 TiB 38 TiB 49.82
TOTAL 75 TiB 38 TiB 38 TiB 38 TiB 49.82
--- POOLS ---
POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL
cephfs_data 15 128 0 B 0 0 B 0 9.6 TiB
cephfs_metadata 16 32 9.7 MiB 24 30 MiB 0 9.6 TiB
cephfs_ec-data 18 128 119 GiB 30.64k 179 GiB 0.60 19 TiBHi,
It's always a good idea to look at the current documentation: https://docs.ceph.com/en/quincy/cephfs/createfs/ which does talk about EC pools in the first section (including the performance issue and what to do about it) and last section.
I guess because you told the Ceph FS to use the EC pool for that directory: https://docs.ceph.com/en/quincy/cephfs/file-layouts/#adding-data-pool-to-file-system ? What if you create files outside of the
ec directory?The issue has been resolved, because the pull request adding the warning was merged: https://github.com/ceph/ceph/pull/31494
It's always a good idea to look at the current documentation: https://docs.ceph.com/en/quincy/cephfs/createfs/ which does talk about EC pools in the first section (including the performance issue and what to do about it) and last section.
If I copy data to the roof of the FS it shows in the default pool.
My question was about the mention in the documentation that even when using only the attached EC pool, some inode data would still be stored in the root pool.
But it's possible that kind of usage does not show in the ceph df or there might be another cosmetic bug in ceph
Nevertheless, thank you again for pointing me to that solution. It works very well so far