Setting Up single Shared FC LUN - multipath -ll shows both Shared and boot lun

[marowens]

I'm setting up a test proxmox lab. It's been a minute but what was new this time was I'm using block level LUNs from a Pure Flash Array. My UCS blades do not have local storage so my service profile was setup properly to access the Pure Array. All configurations were done on the Pure and booting and installing to SAN was successful. Once I made it to creating the Shared LUN I was met with multipathing and what needs to successfully configure each node to get them set to see the LUN.

Most articles were for iSCSI whereas I'm using FC. Finding and understanding the /etc/multipath.conf file is a challenge and I got both from Pure's doc (i think ) When I thought I had completed the steps I ran #multipath -ll and got nothing. I then edited the file with my first find from Pure- restarted multipathd and i got my output. Next was to try 'pvecreate /dev/mapper/mpatha - it failed. I then found the other multipath.conf from Pure and replaced my file with this one. Now my output shows both the SAN Boot LUN and the Shared LUN. Same results trying to run the pvcreate.

reading more and trying to see what this is doing I realized the second multipath file created an exception for the pure array whereas the other didn't. I think this is why I'm seeing both LUNS on each node. I replaced the multipath.conf file with the original and rebooted all 4 nodes (BTW if I didn't state it the cluster was created and all 4 nodes added prior to starting this configuration). I also found on Pure's site to run pvcreate with the Pure's device ID. I did this and successfully created the LUN then refreshed each node and all 4 successfully see the 10T shared LUN.

My issue - well when I run multipath - ll, on each node I see the nodes boot lun and the shared lun. I don't know if it's normal as all output I've seen assumes local boot and a shared iscisi or fc LUN.


screenshots of outputs below

File from Pure to create /etc/multipath.conf Here, I missed the BOLD TOP instructions to use the device ID when doing the pvcreate (rookie mistake)




First output of the above multipath.conf configuration...



Second try at the /etc/multipath.conf (again from PURE)

I can't find the file now but I know it created a blacklist exception for the VENDOR PURE. after reloading multipathd i get this...



Keep in mind the multipath.conf was restored to the original and I still see both LUNs



The ultimate question (where the answer isn't '42') is this normal or should that boot LUN not be seen?

Thanks, in advance

 

[bbgeek17]

Hi @marowens , welcome to the forum.

First thing - do everyone a favor and do not use screenshots, or avoid them if at all possible. Pasting basic text and using CODE tags from the edit box menu is much easier to read.

Second, did you install pure Debian with PVE packages on top, or did you use PVE ISO?
Does your boot LUN have multiple paths? If so you should use them! PVE ISO currently does not provide a way to take advantage of boot time multipath, so you should use Debian install ISO and follow the procedure to use multipath for boot. Followed by installing PVE packages on top.

Even if your boot LUN only has single path (why would it?), you can still make use of multipath. This will cover you if you add second path later.

There is nothing inherently wrong with having MP control your boot LUN, as long as you use correct device names everywhere. If it bothers you - exclude it from MP config.

None of this is PVE specific. You should decide on the path (pun intended) you want to take, and then follow the Pure/Debian install and configuration guides.

Cheers

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[marowens]

I'll take note of the screenshot issue. Thanks

I installed PVE 9.2.1 on bare metal - SAN Boot obviously.

When you're flying in the dark it's about seeing output change after creating the first /etc/multipath.conf file. Essentially, not understanding that when I did the pvcreate and it failed that replacing the mpatha with the device ID I would have never modified the multipath.conf file from the original I created. I did back it up first prior to replacing it and after i restarted services and running the multipath -ll afterwards I now see both.

For me if originally I see only my shared LUN, then change the file and now see both boot and shared LUNs the question is WHY?

If you're saying that seeing both is fine, okay, But seeing only one, the Shared LUN, leaves zero room for error when running the pvcreate /dev/mapper/<device-id>