Performance with KRBD and Writeback.

Jakest

Active Member
Jan 3, 2020
10
2
43
37
We've been doing some testing with KRBD and Writeback, and have noticed -massive- performance gains with the two paired in Windows Server 2019 and OpenBSD guests.

Proxmox 6.3
22 2tb SSD OSD's
Ceph 14.2.22

I've attached crystaldiskmark screenshots of the latency in the VM.

KRBD No Writeback:
KRBD no writeback.jpg

KRBD Writeback:
KRBD writeback.jpg

libRBD No Writeback:
libRBD no writeback.jpg

libRBD Writeback:
libRBD writeback.jpg

These were all tested on the same pool, VM shut down and restarted between each test.

The read and write #'s show the same huge contrast.

KRBD Writeback:
KRBD Writeback Throughput.jpg

libRBD Writeback:
LIBRBD Writeback throughput.jpg


The magic is definitely somewhere in the krbd and writeback combo.


Just wondering if anyone else has seen this crazy difference, as well as wanted to let people know that playing with these things can make a huge difference.
 
Thanks for the idea! I've been tracking some hanging reads & writes in my setup (which I still think are mostly a swap file issue on one server) and this has given me some new ideas to check.