Shared Storage (Fibre-Channel)

[cloudguy]

Good Day,

We are building a few clusters using Proxmox 2.0 RC1 (non prod evaluation / development) and are wondering what the best (and Proxmox recommended) approach is to configure SAN-storage.

Based on what I've seen in Proxmox we really have a few options:

1. use LVM2 storage on shared disk -- would like to hear some experiances with this one, as my understinding is that it's not recommended to use plain LVM2 volume groups on shared disk between multiple systems.
2. use CLVM -- is there a document / procedure published for Proxmox in getting this setup?
3. use a common mount point for all cluster nodes (I guess thats what the directory option is for when configuring storage in the GUI). The intent is that a cluster file system such as OCFS2 would be mounted under this directory.

Thank you.

 

[e100]

I used DRBD and for two nodes is basically like a shared disk.
LVM2 has worked well in Proxmox 1.9 and in 2.0
I think I can answer your points 1 and 2.

I also feel like cLVM should be used and the first 2.0 beta even had cLVM.
But snapshots would not work and some other issues so it ended up going away.

Creating LVM volumes on two nodes at the same time, that might be bad.
Pretty sure Proxmox prevents this with how the code works.

I have had problems doing snapshot backups on two nodes at the same time, so I do not do that.
Other than that it seems to work just fine.

 

[cloudguy]

There's a shared option to select when specifying a volume group. Volume group curated with vgcreate (vg name) (device). I'm hoping there's a little more code behind that checkbox. Wondering I anyone can advise the best practice for lvm based shared storage in v2.0.


Thanks.

 

[dietmar]

1. use LVM2 storage on shared disk -- would like to hear some experiances with this one, as my understinding is that it's not recommended to use plain LVM2 volume groups on shared disk between multiple systems.

We usually use this setup. The PVE internal locking system makes sure that only on VM access a LV at at time.

But there is no locking protection if you want to use lvm tools on the command line. You need clvm in that case.

2. use CLVM -- is there a document / procedure published for Proxmox in getting this setup?

Just set locking_type in /etc/lvm/lvm.conf, the mark VG as clustered.

3. use a common mount point for all cluster nodes (I guess thats what the directory option is for when configuring storage in the GUI). The intent is that a cluster file system such as OCFS2 would be mounted under this directory.

I consider that much to complicated, but some people use gfs2 (ocfs2 is not really supported).

 

[cloudguy]

Just set locking_type in /etc/lvm/lvm.conf, the mark VG as clustered.

I know there are two options (type 2 = built in locking, type 3 = external locking shared library) -- any suggestions which one would work best in a Proxmox environment?

For some reason when I tried to simply set the locking type (to 2) in an attempt I did last week CLVM would not start. I'll give it another attempt on a freshly built cluster tomorrow, and post any issues.

Thanks.

 

[dietmar]

I know there are two options (type 2 = built in locking, type 3 = external locking shared library) -- any suggestions which one would work best in a Proxmox environment?

Those numbers are wrong! You should use type 3, which uses built-in clustered locking (type 2 is external shared lib)

For some reason when I tried to simply set the locking type (to 2) in an attempt I did last week CLVM would not start. I'll give it another attempt on a freshly built cluster tomorrow, and post any issues.

Well, type 2 is external lib, so this will not work.