iscsi multipath problem

GoatMaster · Nov 17, 2017

Hey,

I connected my iSCSI target to the node with multipath.

But I don't get it how can I give the storage now to proxmox to use it for virtual machines ?

adamb · Nov 17, 2017

I typically put LVM ontop of iscsi and then present that to Proxmox.

https://pve.proxmox.com/wiki/Storage:_LVM

GoatMaster · Nov 17, 2017

adamb said:
I typically put LVM ontop of iscsi and then present that to Proxmox.

https://pve.proxmox.com/wiki/Storage:_LVM

I understand that but how to do it with a multipath iscsi target ?

I am not able to add it over the gui.

adamb · Nov 17, 2017

GoatMaster said:
I understand that but how to do it with a multipath iscsi target ?

I am not able to add it over the gui.

You have to put lvm ontop of multipath.

pgcreate /dev/mapper/mpath# (Not sure on your path)
vgcreate nameofvg /dev/mapper/mpath#

Then you can add your new LVM to the gui with "nameofvg".

GoatMaster · Nov 17, 2017

adamb said:
You have to put lvm ontop of multipath.

pgcreate /dev/mapper/mpath# (Not sure on your path)
vgcreate nameofvg /dev/mapper/mpath#

Then you can add your new LVM to the gui with "nameofvg".

Okay I understand.

But..
pgcreate: command not found

adamb · Nov 17, 2017

GoatMaster said:
Okay I understand.

But..
pgcreate: command not found

Ooops. "pvcreate"

GoatMaster · Nov 17, 2017

adamb said:
Ooops. "pvcreate"

everything worked so far, but I can not add the lvm with help of the GUI.

I have three nodes in the cluster do I need to add the iscsi target first to all nodes?

adamb · Nov 17, 2017

GoatMaster said:
everything worked so far, but I can not add the lvm with help of the GUI.

I have three nodes in the cluster do I need to add the iscsi target first to all nodes?

I need a little bit more information. What error is displayed when trying to add the storage from the GUI?

Datacenter -> Storage -> Add -> LVM

GoatMaster · Nov 17, 2017

adamb said:
I need a little bit more information. What error is displayed when trying to add the storage from the GUI?

Datacenter -> Storage -> Add -> LVM

ID: Test
Base storage: Existing volume groups
Volume group: (drop down which only shows pve)

adamb · Nov 17, 2017

GoatMaster said:
ID: Test
Base storage: Existing volume groups
Volume group: (drop down which only shows pve)

Provide the output of the following commands if possible.

pvdisplay

vgdisplay

GoatMaster · Nov 17, 2017

adamb said:
Provide the output of the following commands if possible.

pvdisplay

vgdisplay

pvdisplay

Code:

  --- Physical volume ---
  PV Name               /dev/mapper/proxmox-test
  VG Name               proxmox-test
  PV Size               100.00 GiB / not usable 0
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              25601
  Free PE               25601
  Allocated PE          0
  PV UUID               HherM0-xvly-6PL0-4F2L-ujhw-zkUu-vIuXIe

  --- Physical volume ---
  PV Name               /dev/nvme0n1p3
  VG Name               pve
  PV Size               744.96 GiB / not usable 3.82 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              190709
  Free PE               4055
  Allocated PE          186654
  PV UUID               yh29Em-oqbZ-8l1I-i21f-zqtX-i2Cb-jm9pnl

vgdisplay

Code:

  --- Volume group ---
  VG Name               proxmox-test
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               100.00 GiB
  PE Size               4.00 MiB
  Total PE              25601
  Alloc PE / Size       0 / 0
  Free  PE / Size       25601 / 100.00 GiB
  VG UUID               LlRTbE-bQJa-Fqr2-dg9D-9vI2-TMa7-YQvoD7

  --- Volume group ---
  VG Name               pve
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  7
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                3
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               744.96 GiB
  PE Size               4.00 MiB
  Total PE              190709
  Alloc PE / Size       186654 / 729.12 GiB
  Free  PE / Size       4055 / 15.84 GiB
  VG UUID               dSPJfR-3mjc-9BWl-PEv3-hLDl-3kY2-YtWjR9

I want to add the proxmox test to have it in the GUI.

adamb · Nov 17, 2017

GoatMaster said:

pvdisplay

Code:

  --- Physical volume ---
  PV Name               /dev/mapper/proxmox-test
  VG Name               proxmox-test
  PV Size               100.00 GiB / not usable 0
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              25601
  Free PE               25601
  Allocated PE          0
  PV UUID               HherM0-xvly-6PL0-4F2L-ujhw-zkUu-vIuXIe

  --- Physical volume ---
  PV Name               /dev/nvme0n1p3
  VG Name               pve
  PV Size               744.96 GiB / not usable 3.82 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              190709
  Free PE               4055
  Allocated PE          186654
  PV UUID               yh29Em-oqbZ-8l1I-i21f-zqtX-i2Cb-jm9pnl

vgdisplay

Code:

  --- Volume group ---
  VG Name               proxmox-test
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               100.00 GiB
  PE Size               4.00 MiB
  Total PE              25601
  Alloc PE / Size       0 / 0
  Free  PE / Size       25601 / 100.00 GiB
  VG UUID               LlRTbE-bQJa-Fqr2-dg9D-9vI2-TMa7-YQvoD7

  --- Volume group ---
  VG Name               pve
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  7
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                3
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               744.96 GiB
  PE Size               4.00 MiB
  Total PE              190709
  Alloc PE / Size       186654 / 729.12 GiB
  Free  PE / Size       4055 / 15.84 GiB
  VG UUID               dSPJfR-3mjc-9BWl-PEv3-hLDl-3kY2-YtWjR9

I want to add the proxmox test to have it in the GUI.

Looks like the vg doesn't have any space to allocate

Alloc PE / Size 0 / 0

Can you provide the exact pvcreate and vgcreate commands you ran? multipath -ll output might help also.

GoatMaster · Nov 17, 2017

Code:

pvcreate /dev/mapper/proxmox-test
vgcreate proxmox-test /dev/mapper/proxmox-test

Code:

proxmox-test (3648171379389d68145a0d5af6e6f6d9a) dm-5 EQLOGIC,100E-00
size=100G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
  |- 8:0:0:0 sdc 8:32 active ready running
  `- 7:0:0:0 sdb 8:16 active ready running

adamb · Nov 17, 2017

GoatMaster said:

Code:

pvcreate /dev/mapper/proxmox-test
vgcreate proxmox-test /dev/mapper/proxmox-test

Code:

proxmox-test (3648171379389d68145a0d5af6e6f6d9a) dm-5 EQLOGIC,100E-00
size=100G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
  |- 8:0:0:0 sdc 8:32 active ready running
  `- 7:0:0:0 sdb 8:16 active ready running

My mistake, Alloc PE / Size 0 / 0 is what has been used. Honestly everything your doing looks correct.

What if you delete the lvm's and try creating a file system right on the multipath device then mounting it?

vgremove proxmox-test
pvremove /dev/mapper/proxmox-test
mkfs.ext4 /dev/mapper/proxmox-test
mkdir /mnt
mount /dev/mapper/proxmox-test /mnt

GoatMaster · Nov 17, 2017

adamb said:
My mistake, Alloc PE / Size 0 / 0 is what has been used. Honestly everything your doing looks correct.

What if you delete the lvm's and try creating a file system right on the multipath device then mounting it?

vgremove proxmox-test
pvremove /dev/mapper/proxmox-test
mkfs.ext4 /dev/mapper/proxmox-test
mkdir /mnt
mount /dev/mapper/proxmox-test /mnt

It is working now.

But for my understanding how is the best practice to handle this in a ha cluster with 3 nodes.
Do I add this iscsi target now also to the two other nodes?

adamb · Nov 17, 2017

You got it, also make sure the storage is set as shared.

GoatMaster · Nov 17, 2017

Perfect, thanks for your help!

GoatMaster · Nov 20, 2017

adamb said:
You got it, also make sure the storage is set as shared.

It is not possible to mount the storage on the two other nodes.

Search

Search

iscsi multipath problem

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

adamb

Famous Member

GoatMaster

Active Member

GoatMaster

Active Member