Croit have new nvme OSD for preview

Hi all,

We’ve released a preview of FlashStore, croit’s Rust-based Ceph OSD for NVMe. It’s a drop-in replacement for the standard OSD. We think it could make a big difference to Ceph performance in hyper-converged Proxmox setups, so we’d really value feedback from Proxmox users.

It’s packaged for several distributions, including Debian Trixie, which is what Proxmox VE 9 is based on. If you’re interested, please give it a try and let us know how you get on.

Packages: https://github.com/croit/ceph-rust/releases
Docs: https://croit.github.io/ceph-rust/

Please note this is a preview release, so we don’t recommend using it in production yet.

It’s also available for testing in the croit product (frontier edition)
 
I wanted to give it a try, but since my homelab is made mostly of junk I intercepted on its way to the trash, my NVME disks don't support the 4K blocks.

I might have missed it in the linked materials, but is there a plan to give this implementation to ceph upstream? It looks like the license would be compatible.
 
I wanted to give it a try, but since my homelab is made mostly of junk I intercepted on its way to the trash, my NVME disks don't support the 4K blocks.

I might have missed it in the linked materials, but is there a plan to give this implementation to ceph upstream? It looks like the license would be compatible.
We’re developing this in the open because, once it reaches GA, we’ll submit it upstream as a full pull request. As a Ceph Foundation member and an employer of upstream maintainers, we’ve always merged our code upstream, and pushing upstream remains our priority.
 
Last edited:
How does this compare / complement with Crimson/Seastar OSD?
Besides the Rust refactor, some other concepts are similar, like one-thread-one-core or RDMA. Initially both pursue the same goals (i.e. remove software limitations with fast NVMe drives), but seem somewhat redundant. I would like to understand the use case for each, as I'm probably missing something.
Thanks!
 
How does this compare / complement with Crimson/Seastar OSD?
Besides the Rust refactor, some other concepts are similar, like one-thread-one-core or RDMA. Initially both pursue the same goals (i.e. remove software limitations with fast NVMe drives), but seem somewhat redundant. I would like to understand the use case for each, as I'm probably missing something.
Thanks!
Well, most concepts aren't new, it's different angels. :p
Crimson is re-architecting the entire OSD (state machine, peering, scrub, ...), while FlashStore is a drop-in replacement for the current BlueStore OSD. It's a narrower scope, I'd say. FlashStore leans on io_uring (SPDK too), while Crimson does heavy use of SPDK/DPDK. In practice, a FlashStore OSD should be easier to integrate into an existing Ceph cluster. And as FlashStore started its development later, things like GPUDirect and RDMA are onboard from the start (Q1–Q2 2027).

Or simpler, get more out of your NVMe by replacing your OSD daemons and use less memory in the process. :cool:
 
Last edited: