virtio disk caching or not

[stefws]

Wondering if it is good to change caching of a virtio disk in a VM f.ex. from default no-cache to cache=writetrough, when the disk images is stored on Ceph and thus properly not cached in the hypervisor host OS memory as it is not coming from a local disk but rather from host NICs. Though Ceph OSD are cached locally those caches are behind NICs and far/further away from a VM OS cache.

Any advices on usage the various caching mode of VM disks would be nice, URLs of such any one?

 

[spirit]

for ceph, you should use cache=writeback.

ceph rbd driver implement a feature to aggregate coalesced blocks, on seq write pattern.

 

[stefws]

Okay thanks!

What about write-through?

Write-back and write-back (unsafe) are they different?

 

[spirit]

Specific to ceph rbd driver

rbd_cache is enabled with BDRV_O_NOCACHE flag

if (flags & BDRV_O_NOCACHE) {
rados_conf_set(s->cluster, "rbd_cache", "false");
} else {
rados_conf_set(s->cluster, "rbd_cache", "true");
}

qemu set the flags with

if (!strcmp(mode, "off") || !strcmp(mode, "none")) {
*flags |= BDRV_O_NOCACHE | BDRV_O_CACHE_WB;
} else if (!strcmp(mode, "directsync")) {
*flags |= BDRV_O_NOCACHE;
} else if (!strcmp(mode, "writeback")) {
*flags |= BDRV_O_CACHE_WB;
} else if (!strcmp(mode, "unsafe")) {
*flags |= BDRV_O_CACHE_WB;
*flags |= BDRV_O_NO_FLUSH;
} else if (!strcmp(mode, "writethrough")) {



So, for rbd, cache=none or cache=directsync disable rbd_cache,
all others modes enable the cache (without any difference here, but this is specific to rbd)

 

[stefws]

Right, okay.

At what OS 'abstraction' layer does this form of RBD caching happen: VM, Hypervisor host or Ceph OSD layer?
I would like of course to cache as close to my VM OS preferable in VM OS file cache, which I assume comes naturally inheritent

 

[spirit]

Right, okay.

At what OS 'abstraction' layer does this form of RBD caching happen: VM, Hypervisor host or Ceph OSD layer?
I would like of course to cache as close to my VM OS preferable in VM OS file cache, which I assume comes naturally inheritent

Hypervisor host

(note that the size of this cache is around 32mb if I remember)

 

[stefws]

Okay thanks!

Out of curiosity, now you didn't show the flags for write through though, what are the difference between write back and write through then+


} else if (!strcmp(mode, "writeback")) {
*flags |= BDRV_O_CACHE_WB;
} else if (!strcmp(mode, "writethrough")) {
*flags |= ???
}

 

[spirit]

Okay thanks!

Out of curiosity, now you didn't show the flags for write through though, what are the difference between write back and write through then+


} else if (!strcmp(mode, "writeback")) {
*flags |= BDRV_O_CACHE_WB;
} else if (!strcmp(mode, "writethrough")) {
*flags |= ???
}

block.c
int bdrv_parse_cache_flags(const char *mode, int *flags)
{
*flags &= ~BDRV_O_CACHE_MASK;


if (!strcmp(mode, "off") || !strcmp(mode, "none")) {
*flags |= BDRV_O_NOCACHE | BDRV_O_CACHE_WB;
} else if (!strcmp(mode, "directsync")) {
*flags |= BDRV_O_NOCACHE;
} else if (!strcmp(mode, "writeback")) {
*flags |= BDRV_O_CACHE_WB;
} else if (!strcmp(mode, "unsafe")) {
*flags |= BDRV_O_CACHE_WB;
*flags |= BDRV_O_NO_FLUSH;
} else if (!strcmp(mode, "writethrough")) {
/* this is the default */
} else {
return -1;
}




return 0;
}




no special flag for writethrough.