Real thin provisioning with CEPH

[Pourya Mehdinejad]

Hi guys
We have Proxmox and Ceph as our storage.
I understand that by turning on discard on the VMs and doing fstrim we can reclaim the unused storage.
But my question is why as soon as I create a VM with E.G 200GB storage from CEPH pool, Proxmox will deduct that 200GB from the CEPH POOL.
Even after discard and fstrim if the storage decrease to let's say 30GB, but there is no way I can use that 200GB to provision another VM.
My purpose to achive this is to somehow oversell the storage with caution.
I have attached a screenshot of my CEPH Pool.

 

[Alwin]

The storage shows the configured size, not the actual one. You can check with rbd du <pool>/<disk-name> the used size.

 

[Pourya Mehdinejad]

The storage shows the configured size, not the actual one. You can check with rbd du <pool>/<disk-name> the used size.

I know I can see the actual size using those commands, but the issue is my storage in Proxmox will reduce by the given size, not the actual size.
For example, I have a 1TB Pool and storage of CEPH (RDB) in Proxmox, once I create an empty VM with a 100GB virtual disk, my storage has 900GB left and usable for other VMs.

 

[Alwin]

Proxmox VE doesn't impose a limit on the storage, it is just statistics.

 

[Pourya Mehdinejad]

Proxmox VE doesn't impose a limit on the storage, it is just statistics.

But even if I check my pool used size by 'rados df' it shows the provisioned size not the actual size.

 

[spirit]

I know I can see the actual size using those commands, but the issue is my storage in Proxmox will reduce by the given size, not the actual size.
For example, I have a 1TB Pool and storage of CEPH (RDB) in Proxmox, once I create an empty VM with a 100GB virtual disk, my storage has 900GB left and usable for other VMs.

This is not normal. the space is not reserved and not used until you write datas in your vm.
as proof, you can also create vm disk bigger than available storage in your pool.

Just check the storage disk usage graph, create a vm with a disk, the disk usage space of the storage shouldn't change

 

[tapaskrmahato]

I have two node CEPH Cluster. But recently I observed that it utilize high bandwidth. How to resolve this issue. please help me out.

[MODERATOR EDIT] standalone post made at: https://forum.proxmox.com/threads/ceph-use-high-bandwidth-utilization.59209/

 

[wolfgang]

@tapaskrmahato
A two node ceph cluster makes no sense and is under the minimum requirements.
see https://pve.proxmox.com/pve-docs/chapter-pveceph.html#_precondition

 

[abufrejoval]

The storage shows the configured size, not the actual one. You can check with rbd du <pool>/<disk-name> the used size.

that's great to know, even if it took me a while to understand you quite literally need to follow that format...

But I wonder if the allocated size might be added as a column in the storage/disk view of the GUI...

It's nice that the columns are selecteable, now I'd just want to be able to choose the columns which seem operationallay more meaningful, e.g. 'date' doesn't actually contain any data and logical vs. allocated is important to keep track off as VMs go about 'life'.