I've been running some benchmarks on Windows whose virtio disks are on Ceph. All is well. At one point I enabled the "krbd" option because I wanted to use the rbd pool for containers as well. Out of curiosity I reran the benchmarks on windows and to my surprise the write speeds are simply out of this world. Using ATTO benchmark the write speeds for blocks of 1MB+ the speeds are way higher than what the network bottleneck of 10Gbit would dictate. I also do not see heavy writes on our Ceph monitor. Yet according to ATTO we are able to write at up to 7 to 8GByte/s. IOMeter shows similar unbelievable values.
This all leads me to think that something in the stack between the high-level layer, ATTO (application layer), and the low-level layer, Ceph, is aggressively caching writes. This observation frightens me a lot as it is very unlikely that data consistence is guaranteed.
Is this a known issue that anyone knows of, and/or does anyone know how to investigate this further where the caching is being done?
This all leads me to think that something in the stack between the high-level layer, ATTO (application layer), and the low-level layer, Ceph, is aggressively caching writes. This observation frightens me a lot as it is very unlikely that data consistence is guaranteed.
Is this a known issue that anyone knows of, and/or does anyone know how to investigate this further where the caching is being done?
