Is it possible to use LVM on many shared iSCSI LUNs?

[dualamd]

Hi,

I have a small Proxmox cluster, and recently acquire 2 old SAN boxes.
I would like to use all available storage space of those SAN boxes.

Each of those SAN boxes exports a iSCSI LUN to all Proxmox nodes.
I have created a LVM volgroup on a single iSCSI LUN, combined with multipath; everything is running fine [1 volgroup <=> 1 multipath <=> 1 LUN].

I would like to ask: whether those cases are cluster-safe:

1. Using only a single volgroup, contain all iSCSI LUNs.
Reason:
_ simpler in management GUI.
_ VM can have a huge disk that spanning on many LUNs.
_ somewhat improve performance, LUN X on a controller A, LUN Y on controller B, LUN Z on controller A ...
If i use "pvmove" command on a node, is it cluster-safe?
[1 volgroup <=> many multipaths <=> many LUNs]

2. Moreover, if i create a raid0 LV, spreading its Physical Extends on all PVs of a volgroup, then assign than LV to a high IOPS VM, is it cluster-safe?
[1 big LV raid 0 <=> 1 volgroup <=> many multipaths <=> many LUNs]

3. Instead of letting LVM manage raid, i could use linux mdadm.
A LVM volgroup on /dev/md0 on many iSCSI LUNs. Is it cluster-safe? (md scheme maybe linear or raid0)
[1 volgroup <=> 1 /dev/md0 <=> many multipaths <=> many LUNs]

Best regards.

 

[LnxBil]

1. Using only a single volgroup, contain all iSCSI LUNs.

Actually, you're creating a RAID0 over two SANs? That sounds not good. Use a VG for each SAN, then a failure will not kill everything.

is it cluster-safe?

IIRC, most LVM operations are not cluster safe, because clvmd is not running per default. PVE does the syncing manually and everyone has to go through their API to get things done in a safe manner.

 

[dualamd]

Yes, it would be big trouble but i think it's worth a try.
For majority of VMs, standard layout 1 VG / 1 LUN is ok.
But there are some big Postgres VMs, having higher IOPS than 1 old cheap SAN can provide.

Is there any official document by Proxmox on how they implement LVM syncing without clvmd?
Thanks.

 

[LnxBil]

But there are some big Postgres VMs, having higher IOPS than 1 old cheap SAN can provide.

That depends on the SAN and disks of course. We still have an EVA6400 laying around in our basement with a 1,5 racks full of shelves and disks etc. The performance of this beast is still very good, but extremely power hungry.

Is there any official document by Proxmox on how they implement LVM syncing without clvmd?

I don't know if it's writting somewhere, but you're golden as long as you use the GUI.