Ceph with shared consumer NVMe for DB and Wall

[adriano_da_silva]

Let's see what an interesting product...

I use Ceph (bluestore) with several nodes, but each one of them has only 1 HD. I want to add more hard drives to each node, but I see that the performance is not satisfactory for virtual machine disks on HD OSD's. So I'm looking to add NVMe's for walls and DBs. However, I have come across information that consumer NVMe's are not good, because they don't have supercaps, which served as a UPS for NVMe, as Ceph uses fsync() calls all the time.

Due to the fact that the price of the enterprise SSD is very expensive.

As I use legacy hardware, my motherboard doesn't have native NVMe connector (on any of my servers). So I'm thinking about using a small and common NVMe adapter for PCIe 4X slots, as it is very cheap hardware.

So I found a very interesting specific adapter! Would this type of adapter help in this case to use with a consumer NVMe?

Apparently, this adapter has a real UPS before Drive NVMe. But would it have the ability to intercept Ceph's fsync() calls and not send them to NVMe, controlling and sending the signal only when it's really needed?

See the adapter discussion here: https://www.truenas.com/community/threads/x4-pcie-to-nvme-adapter-with-supercapacitor.58810/

 

[adriano_da_silva]

To be clearer, the adapter I mentioned above is quite cheap and allows the use of consumer NVMe in PCIe 4X slots, in addition to containing a super capacitor which, according to the tests in the link above, works perfectly as a UPS directly before the NVMe, keeping it running for several seconds after a power failure.

Would this help a consumer NVMe do the same job as an SSD with built-in supercaps when we talk about use in Ceph - with fsync() calls? Or would it be possible to have some configuration that reduces fsync() calls in Ceph?

 

[Tsvyatko Kriviradev]

Hello @adriano_da_silva how did you solve that problem?
Can you share good consumer nvme devices that are working good for you?

 

[LnxBil]

Would this help a consumer NVMe do the same job as an SSD with built-in supercaps when we talk about use in Ceph - with fsync() calls? Or would it be possible to have some configuration that reduces fsync() calls in Ceph?

No, it would not. fsync speed will not improve with the cap, it just won't fuck up for consumer NVMe after a power loss.

Just buy the enterprise hardware if you want enterprise performance or buy twice