Hot to add LVM volume group on slave cluster node?

[klabauter]

How to add a LVM volume group on a slave cluster node?

I have a 3 Node Cluster Setup
-N1 master
-N2 slave
-N3 slave

N1-N2 have an shared DRBD device running in primary/primary mode as a LVM volume group.
On the Master (N1) I could add the LVM VG easily and it works fine even kvm live migration to N2.

When I try to add in Proxmox Webgui(on N2 or N3) the LVM VG of N2-N3 he says "no write permission" which he always says when I'am not on the master webgui.

My problem is, that on the Master the LVM volume group not exist.
I am running Proxmox 1.5 with Kernel 2.6.32

 

[udo]

Hi,
to use clusterwide volume groups their must by accessible on the master (and on the other nodes too). Therfore external storage (iScsi + FC, nfs) are the best way for lvm-storage.

Udo

 

[klabauter]

thank you for your reply.

So I have to configure the drbd volumes to be accessible by all nodes especially the master via iScsi/Nfs and then adding a lvm volume group?
It would add another complexity level to the storage.

Right now I have 4HDDs->Raid10->LVM->DRBD->LVM->PVEs.

I have no experience with NFS/iScsi but I think I have to configure a HA environment with this services than, right?
When one Node failed, I want to be able to start the PVEs on another Server without data loss and also livemigrate from one node to another(clusterwide is not needed hardly)

It would be very nice if someone could give me a tip to solve this issue propper(without a SAN).

But i would prefer that proxmox simply knowes that there is a storage which is just accessible for Node 2 and 3.

 

[dietmar]

Sorry, but AFAIK PVE and DRBD onyl work with 2 cluster nodes - the current impl. has problems when you use more than 2 cluster nodes and DRBD.

 

[klabauter]

This would be a nice feature I think. When you can have multiple pairs in your Cluster with drbd.
When then you can create this drbd volumes with the Web Gui it would be perfekt!
Hopefully we can see somethink like this on the roadmap sometime.

 

[tito]

I have a similar problem and i can't find any similar responses on the forum.

I have 4 servers, two of them forming one cluster with DRBD and Volume Group "A", and the others two with another DRBD resources with Volume Group "B".
I added the last two servers two the cluster, so that i can manage the four servers from the master server.

But... i can't add the volume group "B" to the storage section of the master. Thus, i can't use volume group "B" to install KVM machines.

My question is: Is it possible to add volume group "B" in the master web GUI? And how?
Last and probably most important: will pve 2.X affect the question i made before?

Thanks for your great job !

 

[udo]

I have a similar problem and i can't find any similar responses on the forum.

I have 4 servers, two of them forming one cluster with DRBD and Volume Group "A", and the others two with another DRBD resources with Volume Group "B".
I added the last two servers two the cluster, so that i can manage the four servers from the master server.

But... i can't add the volume group "B" to the storage section of the master. Thus, i can't use volume group "B" to install KVM machines.

My question is: Is it possible to add volume group "B" in the master web GUI? And how?
Last and probably most important: will pve 2.X affect the question i made before?

Thanks for your great job !

Hi,
i had a time ago a third node in a cluster which don't have the drbd-storage of the other. I use simply a usb-stick on which i have created the same volumegroup (name). After that i can use all nodes (but of course not the drbd-vg on the third node).
Perhaps this dummy-storage will also work on the master?
With pve2.x the storage modell should be more flexible.

Udo

 

[Mario Wolff]

Hi all,
we are working with a three node cluster with three drbd's sharing storage between two nodes (primary- and backup-node). On each Node i've added:

dd if=/dev/zero of=/tmp/pvdummy0 bs=512 count=1024
pv=$( losetup -s -f /tmp/pvdummy0 )
pvcreate $pv
vgcreate <missing vg name> $pv

to /etc/rc.local. This will create the smallest possible physical volume as a file in /tmp and everything is working as expected!

Regards,
Mario

PS: It would be great to have a hook in pvemirror to create missing vg's that way when /etc/pve/storage.cfg is updated. A mirror of all config files, maybe under /etc/pve/<nodename or nodeid> would also be great! Right now we rsync /etc/qemu-server (kvm only pve-nodes!) to all other nodes to have the config for a vm on his backup-node.

 

[tito]

Thanks for your great tips.
In resume, you need to have the same Volume Groups on ALL nodes, or at least all of them in the master... and you need to do it manually, there is no PVE official (or correct?) way of doing it yet.

I hope for 2.X they can figure out how to manage this automatically, since it would be very very usefull to have only one node (master) from where you can manage many many nodes. Since doing it by hand would definitly work as you mentioned, but it is a little problem every time we do upgrades and stuffs like that.

Thank you very much!

 

[tom]

we are aware of this shortcoming and it will be addressed in the next major release (2.x).

 

[tito]

That's good news.
Hope to see 2.x soon, although i won't ask when. Can't wait to have it and test it.
Thanks to all proxmox staff.