Possible to use LVM Caching on PVE host?

[Miraculix_de]

500GB SSD with PVE and 8 TB HDD. Wanted to create an LVM cache to be reach better read speeds. But the question is how this works with LVM-thin...

I already asked this question on serverfault.com but got no answers: https://serverfault.com/questions/1...dea-to-use-lvm-thin-together-with-lvm-caching

I rolled back a backup (10G) and observed incredible slow transfer speeds (according to iotop). First, the transfer was very quick, but at ~40% it slowéd down a lot and reached then 3MB/s. I wonder if this is because of the LVM caching? Switched it down now as I'm usure if its working correctly.

What do you think?

 

[LnxBil]

Works like a charm. Haven't used it with LVM-thin, but LVM-thick many years ago. You need a technique that combines the disk and the SSD to one device and then but the LVM on top. Maybe a bit problematic with a PVE install, but in the end it boils down to the used caching technique.

In my experiments, random I/O write went to the cache device and were very fast. The cache also improved regular reads of the same files and the write speed was at least the one from the disk (the slowest device).

 

[Miraculix_de]

It's clear that it works with normal LVM volumes. The question is if it also works with LVM-thin. As you can see in my link, when adding an LVM-Cache to to data disk, it's in fact attached to data_tdata, which is related to thin pool.

Otherwise, when adding a LVM-Cache to a normal LV, the cache is directly atasched to that LV.

Thats why I guess somethin messed up.

 

[Deleted member 205422]

It's clear that it works with normal LVM volumes. The question is if it also works with LVM-thin. As you can see in my link, when adding an LVM-Cache to to data disk, it's in fact attached to data_tdata, which is related to thin pool.

Otherwise, when adding a LVM-Cache to a normal LV, the cache is directly atasched to that LV.

Thats why I guess somethin messed up.

The brackets in the lvs output mean those are not mountable but part of something else. LV-thin acts like VG. You create thinpool as if an LV but then you create LVs within that pool which you specify where you would normally refer to VG. Care to share how you created the data_tdata?

 

[LnxBil]

It's clear that it works with normal LVM volumes.

I don't think I do then ... the caching methods I used created a new device backed by an SSD and a disk. On top of that device, you can put whatever you want including LVM thin and thick, ext4 ...

 

[ness1602]

For proper implementation we need to wait for bcachefs

 

[Miraculix_de]

What I did

#/dev/sda fast SSD
#/dev/sdb slow HDD
pvcreate /dev/sdb1
vgextend pve /dev/sdb1
pvmove -n /dev/pve/data /dev/sda2 /dev/sdb1
# data is now on sdb
lvcreate -L 30G -n data_cache pve /dev/sda2
lvcreate -L 30M -n data_cache_meta pve /dev/sda2
lvconvert --type cache-pool --cachemode writethrough --poolmetadata pve/data_cache_meta pve/data_cache
lvconvert --type cache --cachepool pve/lv_cache pve/data

Please be aware that I wrote this down from my memories. Hope I m,issed nothing.

From here