Proxmox Cluster with Shared Storage

[jogi]

Hallo everybody,

we have built a Proxmox Cluster with a Shared Storage. The storages sync the VMs over a 10Gigabit Lan with DRBD. Over iscsi we export one partition from the store to each Proxmoxhost.
These three Proxmoxhosts work together in a proxmoxcluster.

We constructed this, because one year before the proxmoxroadmap didn't have suche a feature on it.

At the moment we have a productive environment with 23 VMs.

It works very well. Good job proxmox team.

P.S. We can't wait until you release a beta of this "official" proxmoxfeature.

 

[nasenmann72]

Hi jogi,

this sounds very interesting! Could you give some more detailed information about how you have done that, please? I'm looking for a similar solution.

Thanks in advance,
der Nasenmann

 

[jogi]

Hi Nasemann72,

what would you like to know?

2 Storages:

• Adaptec 1680 Controler RAID 6, 16*SAS HDDs with 146 GB

• Debian etchnhalf 64bit
• DRBD 8.2.7 (10Gbit with the other store, syncs in realtime all VM-data from one to the other hosts)

• iscsitartet 0.4.15 (three exports, one for one proxmoxhost)
• LVM2 (Raid6 with three partitions, one for one proxmoxhost)
• 5*1Gbit bonding-device in mode 5 (iscsi export)

3 proxmoxhosts:

• 4*4 Cores
• 32 GB RAM
• 3 Nics in Bonding-Mode 5 (iscsi initiator)
• proxmox 1.1 (friday update to 1.3)

These are short facts of our serverconfiguration. If you are interested in any details, please post it here.

 

[organetic]

I'm looking forward to a configuration like this.
But our idea would be to use DRBD on the proxmox servers, avoiding a SAN.
Or... if possible, we could use SAN as a virtual machine. Is this approach viable?
I have 10 pairs of servers wainting for this.

Regards,
organetic

 

[kaptk2]

I am interested in your setup as well. What does your Proxmox Host network configuration look like? I am having trouble with making vmbr0 to use a bonded interface. Also why use mode 5 over mode 6 or 802.3ad?

 

[xrisse]

Hi Nasemann72,

what would you like to know?

2 Storages:

• Adaptec 1680 Controler RAID 6, 16*SAS HDDs with 146 GB

• Debian etchnhalf 64bit
• DRBD 8.2.7 (10Gbit with the other store, syncs in realtime all VM-data from one to the other hosts)

• iscsitartet 0.4.15 (three exports, one for one proxmoxhost)
• LVM2 (Raid6 with three partitions, one for one proxmoxhost)
• 5*1Gbit bonding-device in mode 5 (iscsi export)

3 proxmoxhosts:

• 4*4 Cores
• 32 GB RAM
• 3 Nics in Bonding-Mode 5 (iscsi initiator)
• proxmox 1.1 (friday update to 1.3)

These are short facts of our serverconfiguration. If you are interested in any details, please post it here.

Hi jogi,

i'm very interested in some details. i have got two clusternodes with 1 TB harddisk on each. my first idea was to use drbd to sync the content of /var/lib/vz. but what about the migration of an virtual machine. pve will read the vm-files on one node an write it to the other node - but both nodes will use the same storage, isn't it?
i'm confused... :-(