iscsi as a distributed (shared) storage on proxmox

[ispirto]

Hello,

What I'm trying to achieve is attaching a single iscsi target to 2 proxmox servers using a clustered file system.

1. Anyone ever did this with proxmox? If so, what clustered file system did you use?
2. If I do this, would I be able to live migrate or use HA with proxmox just like using a NFS share?

Thanks,
Oktay

 

[ispirto]

For people who is wondering, I think it's possible.

What I did was in steps:

1. Create a SCSI target
2. Attach it on proxmox server 1
3. Create a volume group and logical volume on the target
4. Install debian kernel from squeeze backports (Because OCFS2 is not supported at PVE kernel)
5. Install ocfs2-tools
6. Edit cluster.conf for ocfs2
7. Start ocfs2 on both servers
8. Format newly created logical volume in ocfs2
9. Attach iscsi target on proxmox server 2

 

[udo]

Hello,

What I'm trying to achieve is attaching a single iscsi target to 2 proxmox servers using a clustered file system.

1. Anyone ever did this with proxmox? If so, what clustered file system did you use?
2. If I do this, would I be able to live migrate or use HA with proxmox just like using a NFS share?

Thanks,
Oktay

Hi Oktay,
without cluster FS and lvm instead is the normal doing. The only effort for an cluster fs is the usabality for OpenVZ too (?!).

Udo

 

[ispirto]

Hi Oktay,
without cluster FS and lvm instead is the normal doing. The only effort for an cluster fs is the usabality for OpenVZ too (?!).

Udo

Hello Udo,

Yes, I've noticed this can be done without a cluster fs, too. The key is marking the lvm as shared.

Would this be a problem in the future? Or is it stable?

Basically what I did is:

1. Create a iscsi target with one LUN at the storage server
2. Attach the iscsi target with iscsiadm --mode node --targetname iqn.2013-03.local.openmediavault:shared --portal 192.168.1.50:3260 --login
3. pvcreate /dev/sdb
4. vgcreate sharediscsi /dev/sdb
5. Add LVM volume group at Web UI and mark as shared.

I suppose you can also attach iscsi target at proxmox without checking "Use LUNs Directly" checkbox in Web UI, then just add a LVM storage in Web UI again by defining the volume group name manually there and proxmox will create the physical volume and volume group at the LUN at the iscsi target.

Is using Web UI to attach iscsi target any different than doing it by iscsiadm command in shell?

I'm asking this because eventually I'll use a SRP (SCST Infiniband) target, add it to all nodes and create pv and vg then add it on the Web UI. Is this approach stable? Any ideas?

Thanks

 

[udo]

Hello Udo,

Yes, I've noticed this can be done without a cluster fs, too. The key is marking the lvm as shared.

Would this be a problem in the future? Or is it stable?

Hi,
lvm-storage is very stable

Basically what I did is:

1. Create a iscsi target with one LUN at the storage server
2. Attach the iscsi target with iscsiadm --mode node --targetname iqn.2013-03.local.openmediavault:shared --portal 192.168.1.50:3260 --login
3. pvcreate /dev/sdb
4. vgcreate sharediscsi /dev/sdb
5. Add LVM volume group at Web UI and mark as shared.

I suppose you can also attach iscsi target at proxmox without checking "Use LUNs Directly" checkbox in Web UI, then just add a LVM storage in Web UI again by defining the volume group name manually there and proxmox will create the physical volume and volume group at the LUN at the iscsi target.

right, see also http://pve.proxmox.com/wiki/Storage_Model

Is using Web UI to attach iscsi target any different than doing it by iscsiadm command in shell?

I'm asking this because eventually I'll use a SRP (SCST Infiniband) target, add it to all nodes and create pv and vg then add it on the Web UI. Is this approach stable? Any ideas?

Thanks

don't know - I use Infiniband for DRBD only...

Udo