[SOLVED] LVM on iSCSI (Multipath)

[Samaritan]

Hey folks,

I have some issues creating a LVM on iSCSI. I am new to proxmox and just started to create a homelab with it.
I have a Qnap NAS. QNAP shows the connection is correctly established.


For proxmox I created a cluster with two nodes. Both show the iSCSI as active but the lvm on top of iSCSI as "status:unknown".



So I started to analyze and found this three commands for CLI:


It seems to me that the LVM is not correctly created. I added it via GUI and there was no error. Still... in the storage.cfg it appears correct:


So I tried to add a new LVM:


I also rebooted both nodes and following services:
systemctl restart pvedaemon
systemctl restart pvestatd
systemctl restart pveproxy
systemctl restart corosync

I also started the whole process from beginning. Deleted the iSCSI and LVM on proxmox and also on QNAP and started all over. Same issues.
From here I do not know what to do. Any help would be very appreciated!

 

[fweber]

Hi, is it possible that there are multiple paths (i.e., portals) to the SAN? Can you send the output of iscsiadm -m session? If it shows more than one session to the same target via different portals, LVM will see the PV on the LUN multiple times and refuse to use it -- you can check the output of vgscan -vvv 2>&1, it should show a warning describing this. If this is the case, you can set up multipath on the host, which will create a multipath device that will be usable by LVM, see the wiki [1].

[1] https://pve.proxmox.com/wiki/ISCSI_Multipath

 

[Samaritan]

root@pve:~# iscsiadm -m session
tcp: [1] 192.168.1.30:3260,1 iqn.2004-04.com.qnap:ts-451plus:iscsi.qnap2pve.2e0183 (non-flash)
tcp: [2] 192.168.1.31:3260,1 iqn.2004-04.com.qnap:ts-451plus:iscsi.qnap2pve.2e0183 (non-flash)

You are correct. This is the case here. Thank you very much!
As I have two node I will need to install and configure it on both nodes. Am I correct?

 

[fweber]

root@pve:~# iscsiadm -m session
tcp: [1] 192.168.1.30:3260,1 iqn.2004-04.com.qnap:ts-451plus:iscsi.qnap2pve.2e0183 (non-flash)
tcp: [2] 192.168.1.31:3260,1 iqn.2004-04.com.qnap:ts-451plus:iscsi.qnap2pve.2e0183 (non-flash)
You are correct. This is the case here. Thank you very much!

Seems like this is indeed the problem. Even though you only added the portal 192.168.1.30:3260 to the PVE storage config, the SAN actually advertises multiple available portals, and the Open-iSCSI initiator automatically connects to all of them.

As I have two node I will need to install and configure it on both nodes. Am I correct?

Exactly.

 

[Samaritan]

Sorry to bother you again. Maybe I miss something here.

Just sorted out how to find the device and then the wwid:

root@pve:~# lsblk

NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS

sda                  8:0    0 476.9G  0 disk

├─sda1               8:1    0  1007K  0 part

├─sda2               8:2    0     1G  0 part /boot/efi

└─sda3               8:3    0 475.9G  0 part

  ├─pve-swap       253:0    0     8G  0 lvm  [SWAP]

  ├─pve-root       253:1    0    96G  0 lvm  /

  ├─pve-data_tmeta 253:2    0   3.6G  0 lvm

  │ └─pve-data     253:4    0 348.8G  0 lvm

  └─pve-data_tdata 253:3    0 348.8G  0 lvm

    └─pve-data     253:4    0 348.8G  0 lvm

sdb                  8:16   0     1T  0 disk

sdc                  8:32   0     1T  0 disk



root@pve:~# /lib/udev/scsi_id -g -u -d /dev/sdb

36e843b6a0d38528d812dd4686d8704df

root@pve:~# /lib/udev/scsi_id -g -u -d /dev/sdc

36e843b6a0d38528d812dd4686d8704df

root@pve:~#

It seems wierd to me that the wwid for both is identical. Shoudn't there be two wwids? One for each path?

 

[fweber]

The identical WWIDs are expected, because the WWID uniquely identifies the LUN (and does not take into account the path to the LUN). As you have two paths to the LUN, you see two block devices for the LUN, but the two block devices report the same WWID.

 

[Samaritan]

Thanks for explaining. Works perfect!

 

[fweber]

Great to hear! Could you please mark the thread as SOLVED (by editing the first post)?