Maximum VM Disk Size for Ceph RBD

[chrone]

Hi Proxmoxers,

I could not find the docs regarding maximum single RBD size for KVM.

Could you please shed a light on the maximum size of a single Ceph RBD disk which Proxmox could handle?

Is it safe to allocate a single disk in Ceph RBD with size like 100TB or more for storing backup files?

 

[Alwin]

AFIAK, there is no limit imposed on rbd images or what qemu can handle. But there are other limitations that need consideration, see the link. https://unix.stackexchange.com/a/33561

For your backup purpose, you could use cephfs, it is also integrated into Proxmox VE.
https://pve.proxmox.com/pve-docs/chapter-pvesm.html#storage_cephfs

 

[chrone]

AFIAK, there is no limit imposed on rbd images or what qemu can handle. But there are other limitations that need consideration, see the link. https://unix.stackexchange.com/a/33561

For your backup purpose, you could use cephfs, it is also integrated into Proxmox VE.
https://pve.proxmox.com/pve-docs/chapter-pvesm.html#storage_cephfs

I see thanks for the links.

The concern with CephFS is its direct and/or synchronous write throughputs are terribly slow compared with RBD on all HDD OSDs. Also, CephFS quota on kernel client only works on kernel 4.17 or above, and only on Mimic Ceph cluster or above.

 

[Alwin]

The concern with CephFS is its direct and/or synchronous write throughputs are terribly slow compared with RBD on all HDD OSDs. Also, CephFS quota on kernel client only works on kernel 4.17 or above, and only on Mimic Ceph cluster or above.

I have seen this for small writes, yes. But for big files (eg. vzdump backups) the speed is up there with RBD. Another possibility is to save the backup locally first and then transfer to cephfs, this can be done with the backup hooks. You can also use the fuse client (has quota support) for cephfs, the storage plugin can handle both variants.

 

[chrone]

I have seen this for small writes, yes. But for big files (eg. vzdump backups) the speed is up there with RBD. Another possibility is to save the backup locally first and then transfer to cephfs, this can be done with the backup hooks. You can also use the fuse client (has quota support) for cephfs, the storage plugin can handle both variants.

Awesome! Thanks, Alwin!