Suggestion: integrate block level SSD caching into the PVE kernel

[gkovacs]

Most of us using Proxmox VE are still tied to rotational storage, be it a single hard drive or a RAID array. Switching to solid state storage for the entire server is usually prohibitely expensive (even more if you want RAID1 or RAID5 redundancy). Although there are hardware-based caching solutions, there is a new breed of software that - coupled with affordable smaller SSD drives - would give a similar result.

There are two options, each with their own strengths and weaknesses:

1. BCache
http://bcache.evilpiepirate.org/

2. Flashcache by Facebook
https://github.com/facebook/flashcache/blob/master/doc/flashcache-sa-guide.txt

More information is available here:
http://www.quora.com/What-are-the-differences-between-Flashcache-and-bcache-projects
http://hightechsorcery.com/2010/07/using-an-ssd-as-a-cache-device/

What I suggest is that one of these should be integrated into the Proxmox VE kernel, preferably with installer and management GUI support later on. This feature would mean a very nice addition to PVE, I guess many of it's users would welcome it.

Should the development team integrate this patch into the kernel, I volunteer for testing with several (Intel and Sandforce based) SSD drives. If it proves to be stable enough in the lab, we could provide production environment testing as well in heavily used servers.

 

[tom]

Why don´t you just use a raid controller supporting a single SSD as an extended cache? At least, Adaptec and LSI are offering this already.

 

[gkovacs]

Why don´t you just use a raid controller supporting a single SSD as an extended cache? At least, Adaptec and LSI are offering this already.

Couple of reasons:

1. Cost-effectiveness
- Adaptec sells you a rebranded off-the-shelf SSD at a huge price premium
- a bit older but otherwise great controllers are not upgadeable, so we would have to buy new ones (an extra 600-700 euros per server compared to 100-150 per SSD)

2. Choice and customizability
- would use different sized SSD cache drives in different servers, depending on array size and workload
- would configure the caching differently, depending on workload
- would be able to cache a single HDD proxmox installation, without the need for an expensive disk array

3. Curiosity and the love of bleeding-edge technology
I like to test and use newly-developed, custom made, configurable solutions much more than ordering boxes from the Internet. This is the main reason I've started using Proxmox at beta stage 3 years back, as you guys married several promising technologies into one easy-to-use product. I think it's time to continue this tradition, and not revert to the "solve every problem with more money" route.

I reckon your user base would welcome to test a feature like this, as it would greatly enhance the appeal of PVE as an integrated product.

Does it require great effort to integrate a patch like this into a test kernel?

 

[linum]

Indeed such a block level cache would be nice. The solutions offered by Adaptec and LSI are not that "hot" as a single SSD and something like linux taking control (more knobs to play with).

 

[tom]

It would be nice to play with it. but nothing for a stable Proxmox VE platform in the moment- but feel free to do some testing and report back.

 

[gkovacs]

It would be nice to play with it. but nothing for a stable Proxmox VE platform in the moment- but feel free to do some testing and report back.

To be able to test I would need the PVE kernel to include the bcache or the flashcache patch. (I don't think there is much point in testing these in vanilla kernels.)

Is the PVE kernel source available for compilation?

 

[tom]

See http://pve.proxmox.com/wiki/Where_to_get_the_sources

 

[jleg]

Couple of reasons:

1. Cost-effectiveness
- Adaptec sells you a rebranded off-the-shelf SSD at a huge price premium

that was true for an older (first?) version, current version neither includes a SSD, nor does it have any restrictions on what SSD to include; we're running several of those with "cheap" OCZ SSDs...
Though caching effects heavily depend on usage profile.
But what is quite nice - you can add more than one SSD, and link or pool it to the RAID instance you want, all during runtime.