Advice for someone looking at using Proxmox (KVM) in our small office

[greavette]

Hello,

We have 15 VM's currently running in our small office. We currently run 3 Host Servers of various sizes to handle our Virtual Machines. All three run VMware Server 2.0. Our office has been humming along very nicely for the past 3 years, but I'd like to make some big changes to our Server environment.

Our budget is allowing us to buy two new Servers and we're looking at two Dell 710's with 32GB of Ram and 3 raid arrays (Raid 1, Raid 10, Raid 1). The reason we have 3 raid arrays is to keep our SQL Server database files on different spindles.

I'm hoping this forum can clear up a few things for us.

• I see there is an .iso image for Proxmox. Do I need to have an O/S already on my Servers before installing this image? If an image is required, would Ubuntu Server LTS edition work well?
• If we wanted to pay for Support to help us set our system up, what would you recommend us buy?
• For High Availability (auto migrate), do we need to have a SAN in place or can we have auto-migrate between the drives on both servers. Another option we are considering is to buy a QNAP device and use NFS to run our VM's but the problem with a NAS would be another single point of failure.
• Would you recommend we convert our existing VMware VM's or should we build Net New and install the software we need.

Any recommendations/comments you wish to provide would be greatly appreciated.

Thank you.

 

[briankwest]

1. Its bare metal so you don't need an os.
2. They offer support on their website its not very expensive.
3. YES CONVERT!

/b

 

[udo]

Hello,

We have 15 VM's currently running in our small office. We currently run 3 Host Servers of various sizes to handle our Virtual Machines. All three run VMware Server 2.0. Our office has been humming along very nicely for the past 3 years, but I'd like to make some big changes to our Server environment.

Our budget is allowing us to buy two new Servers and we're looking at two Dell 710's with 32GB of Ram and 3 raid arrays (Raid 1, Raid 10, Raid 1). The reason we have 3 raid arrays is to keep our SQL Server database files on different spindles.

But which raid is for what? I assume the first raid-1 is the OS (ok), raid-10 for normal VMs? (ok), SQL on the other raid-1? Ok - but i prefer SSD in this case, because the IOPs.

I'm hoping this forum can clear up a few things for us.

• I see there is an .iso image for Proxmox. Do I need to have an O/S already on my Servers before installing this image? If an image is required, would Ubuntu Server LTS edition work well?

pve use debian and the iso contained the full OS allready (like briankwest wrote)

• If we wanted to pay for Support to help us set our system up, what would you recommend us buy?
• For High Availability (auto migrate), do we need to have a SAN in place or can we have auto-migrate between the drives on both servers. Another option we are considering is to buy a QNAP device and use NFS to run our VM's but the problem with a NAS would be another single point of failure.

You need shared storage - mean storage which is available at all nodes, where the VM should be runnable. This can be NFS, SAN (iScsi, FC) or DRBD.
For your config is DRBD an realy nice feature - but of course you need the doubled hdd-space (and take for fast interconnect an 10GB-ethernet-nic (simply crossover) or infiniband-nic for each server).

• Would you recommend we convert our existing VMware VM's or should we build Net New and install the software we need.

Any recommendations/comments you wish to provide would be greatly appreciated.

Thank you.

One thing: It's much better to have a 3-node-cluster (quorum), so if you transfered enough VMs from vmware, you can add one ex-vmware-node with fresh pve to the cluster (of course can't this node access the drbd-volumes, but for quorum and some not-important VMs good enough).

Udo

 

[greavette]

Thanks very much for the reply.

I've been reading up on using the 3-node-cluster (quorum). I have a spare desktop but it doesn't have a VT processor to run any VM's in Proxmox. Could I still use this in my quorum or do I need a third Server as beefy as my other two?

Thank you.

 

[mir]

For your config is DRBD an realy nice feature - but of course you need the doubled hdd-space (and take for fast interconnect an 10GB-ethernet-nic (simply crossover) or infiniband-nic for each server).

I would strongly advice not to use DRBD if the servers are supposed to be used to host a database. We have tried at my work for a moodle installation using Postgresql. The write performance was so bad that we had to turn synchronous commits off.If the database in use is postgresql 9.x I would use postgresql 9.x's build-in support for

streaming replication. In a 2-node setup I would recommend asynchronous replication to a slave. If a 3-node setup is available I would look at the synchronous streaming replication in 9.1​

 

[greavette]

Sorry, I'm not fully versed yet in the features of Proxmox. Are you saying that for our Database VM's, we shouldn't use our QNAP NAS to run the VM. Our database is SQL Server on Windows 2008 Standard.

Should we run our SQL Server VM's on local storage only. We have two identical VM's for our SQL Database where we do log shipping from one DB to the Other. I could run our primary on Server1 and our Backup on Server 2. Weekly (or after Windows updates) I could bring the VM's down and do a backup of the image to our QNAP.

Could we run some VM's locally on our Servers and other VM's on our NAS?

What do you think.

 

[mir]

Sorry, I'm not fully versed yet in the features of Proxmox. Are you saying that for our Database VM's, we shouldn't use our QNAP NAS to run the VM. Our database is SQL Server on Windows 2008 Standard.

Should we run our SQL Server VM's on local storage only. We have two identical VM's for our SQL Database where we do log shipping from one DB to the Other. I could run our primary on Server1 and our Backup on Server 2. Weekly (or after Windows updates) I could bring the VM's down and do a backup of the image to our QNAP.

Could we run some VM's locally on our Servers and other VM's on our NAS?

What do you think.

Install your MS SQL Server on two different VM's and ensure the are provisioned to run on different Proxmox nodes using what ever disk system available. Whether you should run master/slave or master/master between the two database is a matter of taste an experience. But do not use DRBD for database servers running virtualized except the servers get a direct LUN to a SAN mount. A direct SAN mount on the other hand prevents you from live migration in which case you might as well run on physical servers. At the end of the day YMMW.

 

[udo]

Thanks very much for the reply.

I've been reading up on using the 3-node-cluster (quorum). I have a spare desktop but it doesn't have a VT processor to run any VM's in Proxmox. Could I still use this in my quorum or do I need a third Server as beefy as my other two?

Thank you.

Hi,
you need only an 64bit cpu. kvm -machines can't run, but openvz (and you need the node only for quorum).
For power-saving it's better if the node also do some jobs...

Udo

 

[udo]

I would strongly advice not to use DRBD if the servers are supposed to be used to host a database. We have tried at my work for a moodle installation using Postgresql. The write performance was so bad that we had to turn synchronous commits off.If the database in use is postgresql 9.x I would use postgresql 9.x's build-in support for

streaming replication. In a 2-node setup I would recommend asynchronous replication to a slave. If a 3-node setup is available I would look at the synchronous streaming replication in 9.1​

Hi,
I assume that's depends from your drbd-link. With 1GB-connection it's a "no-go", but with 10GB, Dolphin-Nic or Infiniband it's not a real problem. For testing is 1GB ok but in real life you should allways use an faster link (and than is DRBD very nice).
I have an MSSQL and some MySql on drbd-storage without trouble.

Udo

 

[mir]

Hi,
I have an MSSQL and some MySql on drbd-storage without trouble.

Udo​

How many writes per second does your servers generate? On 1 Gbs nic using 8 15k SAS disks in a RAID 10 setup inside an IBM

DS4800*) we were not able to get more than 30-40 write transactions per second which were able to bring a quad core XEON to its ​

knees. Disabling synchronous commits raised it to 120-130 write transactions per second.​

*) This setup is capable of delivering 3500 IOPS.

 

[e100]

I've been using MSSQL in VMs on top of DRBD with Proxmox for over two years.
In every instance the Proxmox/DRBD setup proved faster than the physical machines the VMs replaced.

We used two bonded 1GB ports for DRBD replication at first giving us write speed of about 200MB/sec
Over the last few months we have invested in cheap Infiniband gear and have upgraded 12 of our 14 DRBD nodes so far.
With the IB for replication we are limited to about 700MB/sec(~6Gbps) write speed on the DRBD volume.
The RAID card CPU ability to calculate parity is about 700MB/sec so we can not go faster unless we go to RAID10 or get a faster card.

We use virtio drivers with cache=none to get the best speed and even with that we do not get 700MB/sec from inside the VM but it is more than fast enough for our needs.
You can view some benchmarks of mine in this thread: http://forum.proxmox.com/threads/8540-new-fedora-virtio-win-0-1-22-iso-drive-%28february-2012%29