SSD for High Performance Proxmox Host

[bigfishinnet]

Hello Proxmox forum. First of all thank you for producing such a wonderful piece of software. I have been using it for a abut a year now and just love it. I am more of a Linux beginner and your product offers me very good virtualisation but I would like to move forward with a couple of things.

So I have I am setting up a secure private cloud service for some of my customers and I have a a couple of questions about Proxmox.The situation will be like this.
1) Each customer will have a linux server from zybert.co.uk and this will provide a means of secure access via 2 factor authentication.
2) Files and data for each customer will will reside of the linux server.
3) Each customer will also have a windows server for rdp sessions and general working desktops.
4) The windows server will be virtual machines hosted on a proxmox host.

So the question for you is about Proxmox. I would like to use SSD as this will greatly improve disc access speed but I am a little confused about the config here. Do I install proxmox on a sata raid 10? and then install the vms on SSD's by mounting them in the /etc/fstab. I think it would be best if I had one SSD per customer or do I just one big SSD and run proxmox and run all the vm's off it?In the end the host machine might to be running 2-3 virtual windows 2008 r2 rdp servers.

Many thanks
Stephen

 

[schirpich]

There's a number of ways to do this. But in all honesty I think your best solution is to try them one by one.

A single SSD per VM sounds a bit of an overkill in my opinion. But then again, this really comes down to your specific needs.

If "I" were you I would probably Install Proxmox on a RAID1 at minimum just for redundancy. Nab 4-6 SSD drives and configure them as either RAID10 or RAID6 (depending on what your needs are) and mount the SSD RAID array to a temp location. Shutdown as much as you could and either RSYNC or DD the /var/lib/vz directory to the temporary SSD RAID array. Once completed, modify the fstab file to mount the SSD array as /var/lib/vz/...

This way, you have your speed across the board, and you have your redundancy. Single points of failure always find a way to... well, fail.

In addition, don't forget to configure things like TRIM, I/O Scheduler, possibly journals if you use EXT3 or EXT4, etc. SSD's are a unique breed and require a little bit different TLC. The VM's themselves should ALSO be configured to behave nicely as if they were installed directly onto a SSD in a physical machine. You don't want Winblows needlessly eating away at your SSD with unnecessary writes.

In short TEST, TEST, TEST, and TEST some more. I'm sure as you move through testing you'll find some gotcha's along the way.

And more importantly, report back with your findings. Someone else is sure to come along and have some of the same questions/concerns and you can maybe save them some headaches and time!

Good luck!

 

[bigfishinnet]

There's a number of ways to do this. But in all honesty I think your best solution is to try them one by one.

Hmm that sounds expensive!

A single SSD per VM sounds a bit of an overkill in my opinion. But then again, this really comes down to your specific needs.

Yes you are probably right. I was worried about 2 or 3 busy windows vm's with about 15 users all writing to the disc simultaneously

If "I" were you I would probably Install Proxmox on a RAID1 at minimum just for redundancy. Nab 4-6 SSD drives and configure them as either RAID10 or RAID6 (depending on what your needs are) and mount the SSD RAID array to a temp location. Shutdown as much as you could and either RSYNC or DD the /var/lib/vz directory to the temporary SSD RAID array. Once completed, modify the fstab file to mount the SSD array as /var/lib/vz/...

Would this be Proxmox on RAID 1 SSD or normal spindle discs? Thanks for your advice re the rest.

This way, you have your speed across the board, and you have your redundancy. Single points of failure always find a way to... well, fail.

In addition, don't forget to configure things like TRIM, I/O Scheduler, possibly journals if you use EXT3 or EXT4, etc. SSD's are a unique breed and require a little bit different TLC. The VM's themselves should ALSO be configured to behave nicely as if they were installed directly onto a SSD in a physical machine. You don't want Winblows needlessly eating away at your SSD with unnecessary writes.

Good luck!

Would you enable cache on your RAID for SSD writes - someone advised me against this.

Can you point me to any online guidance regarding this and dont say google! ;-) Two things of note I have come across are

http://www.remote-anywhere.com/07-09-05_raid_performance.pdf
http://www.zdnet.com/blog/storage/why-raid-5-stops-working-in-2009/162

I will report back if it all goes ahead!

Thanks

Stephen

 

[tom]

take a look on http://www.adaptec.com/en-us/_common/maxcache/

never analysed nor I did benchmark´s but maybe an interesting approach for some workloads.

 

[schirpich]

Proxmox install I would probably do on multiple spindles. Just for simplicity sake. Don't quote me on this, but I don't think the system drives play "too much" of a role in VM performance unless maybe backups are stored there or swapping occurs.

Rule of thumb for RAID controller cache is only if you have a battery backup, ESPECIALLY on Production systems. Data loss can occur if data is in cache and power goes out. That data will never see the light of day, pretty much leaving you SOL.

In any case, this should still be a portion of your testing. I can't stress the importance of testing enough. Everybody's application of hardware and software is almost always unique and will produce different results the majority of the time.

But to be perfectly honest on my personal system (no power backup) I use the cache. Knowing it's my own damn fault if something happens. I just can't live without the speed!

In addition, I recently learned a valuable lesson in regards to SSD's. Before buying, spend a day or two reading their support forums. You will get a pretty good overview of common issues and bugs in firmware. I had a OCZ Agility SATA2 SSD fail because of a known firmware bug. All data was lost, only way to restore the drive to working order was to perform a destructive flash. It works great now, but still. 60Gigs of lost data sucks

 

[meto]

First rule of using SSD Drives - always use latest firmware. It's very important, while there is still a lot of they can do to improve performance and durability. It's not like HDD's, when you rarely have firmware updates.

 

[MimCom]

The real answer here is probably something like JFFS2 or UBIfs on raw flash - but good luck finding raw flash now that the so-called value-added services that SSD vendors supply.
http://www.linux-mtd.infradead.org/doc/ubifs.html

 

[isparks_curtis]

In addition, don't forget to configure things like TRIM, I/O Scheduler, possibly journals if you use EXT3 or EXT4, etc. SSD's are a unique breed and require a little bit different TLC. The VM's themselves should ALSO be configured to behave nicely as if they were installed directly onto a SSD in a physical machine. You don't want Winblows needlessly eating away at your SSD with unnecessary writes.

I was just digging around in this forum to see if the Proxmox 1.7 kernel supports trim... so far I haven't been able to confirm one way or another. But, your post does seem to imply that some version proxmox does support it... or are you just assuming that it does?

Thanks,

Curtis