LVM on iSCSI

[MoniPM]

I have set up an iSCSI connection from a Dell server to a NetApp storage device. I have also set up an LVM using the iSCSI connection.

So far, so good.

BUT, the LVM shows up in the PM node with a "?" and showing as not active.

The iSCSI connection works. I can use the LUN directly and create a VM using it but I want the setup an LVM.

I have not been able to find the answer here or elsewhere.

Any advice would be appreciated.

 

[LnxBil]

I can use the LUN directly and create a VM using it but I want the setup an LVM.

How?

I have not been able to find the answer here or elsewhere.

Please post the output of the following commands in CODE tags:

• multipath -ll
• pvs
• vgs
• lvs
• lsblk

 

[MoniPM]

Multipath is not installed.



pvs PV VG Fmt Attr PSize PFree /dev/sda3 pve lvm2 a-- <59.00g 7.25g /dev/sda4 ceph-9ff535a4-bf8c-4052-99ef-c396ede7c8bb lvm2 a-- <871.00g 0 /dev/sdb ceph-b7073224-e6d4-45a4-bbd7-ab4defe512e8 lvm2 a-- <931.00g 0

vgs VG #PV #LV #SN Attr VSize VFree ceph-9ff535a4-bf8c-4052-99ef-c396ede7c8bb 1 1 0 wz--n- <871.00g 0 ceph-b7073224-e6d4-45a4-bbd7-ab4defe512e8 1 1 0 wz--n- <931.00g 0 pve 1 3 0 wz--n- <59.00g 7.25g

lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert osd-block-a83fd1e0-5c7b-4839-a40d-a5f553621201 ceph-9ff535a4-bf8c-4052-99ef-c396ede7c8bb -wi-a----- <871.00g osd-block-4eaf61d2-8938-4131-a2a1-81db59fd5298 ceph-b7073224-e6d4-45a4-bbd7-ab4defe512e8 -wi-ao---- <931.00g data pve twi-a-tz-- <17.56g 0.00 1.58 root pve -wi-ao---- 24.93g swap pve -wi-ao---- 7.25g

lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 931G 0 disk ├─sda1 8:1 0 1007K 0 part ├─sda2 8:2 0 1G 0 part ├─sda3 8:3 0 59G 0 part │ ├─pve-swap 253:1 0 7.3G 0 lvm [SWAP] │ ├─pve-root 253:2 0 24.9G 0 lvm / │ ├─pve-data_tmeta 253:4 0 1G 0 lvm │ │ └─pve-data 253:6 0 17.6G 0 lvm │ └─pve-data_tdata 253:5 0 17.6G 0 lvm │ └─pve-data 253:6 0 17.6G 0 lvm └─sda4 8:4 0 871G 0 part └─ceph--9ff535a4--bf8c--4052--99ef--c396ede7c8bb-osd--block--a83fd1e0--5c7b--4839--a40d--a5f553621201 253:3 0 871G 0 lvm sdb 8:16 0 931G 0 disk └─ceph--b7073224--e6d4--45a4--bbd7--ab4defe512e8-osd--block--4eaf61d2--8938--4131--a2a1--81db59fd5298 253:0 0 931G 0 lvm sdc 8:32 0 931G 0 disk sdd 8:48 0 465.3G 0 disk sde 8:64 0 111.3G 0 disk sdf 8:80 0 4T 0 disk sdg 8:96 0 4T 0 disk root@pms-2:~# pvs PV VG Fmt Attr PSize PFree /dev/sda3 pve lvm2 a-- <59.00g 7.25g /dev/sda4 ceph-9ff535a4-bf8c-4052-99ef-c396ede7c8bb lvm2 a-- <871.00g 0 /dev/sdb ceph-b7073224-e6d4-45a4-bbd7-ab4defe512e8 lvm2 a-- <931.00g 0 root@pms-2:~# ^C root@pms-2:~# vgs VG #PV #LV #SN Attr VSize VFree ceph-9ff535a4-bf8c-4052-99ef-c396ede7c8bb 1 1 0 wz--n- <871.00g 0 ceph-b7073224-e6d4-45a4-bbd7-ab4defe512e8 1 1 0 wz--n- <931.00g 0 pve 1 3 0 wz--n- <59.00g 7.25g root@pms-2:~# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert osd-block-a83fd1e0-5c7b-4839-a40d-a5f553621201 ceph-9ff535a4-bf8c-4052-99ef-c396ede7c8bb -wi-a----- <871.00g osd-block-4eaf61d2-8938-4131-a2a1-81db59fd5298 ceph-b7073224-e6d4-45a4-bbd7-ab4defe512e8 -wi-ao---- <931.00g data pve twi-a-tz-- <17.56g 0.00 1.58 root pve -wi-ao---- 24.93g swap pve -wi-ao---- 7.25g root@pms-2:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 931G 0 disk ├─sda1 8:1 0 1007K 0 part ├─sda2 8:2 0 1G 0 part ├─sda3 8:3 0 59G 0 part │ ├─pve-swap 253:1 0 7.3G 0 lvm [SWAP] │ ├─pve-root 253:2 0 24.9G 0 lvm / │ ├─pve-data_tmeta 253:4 0 1G 0 lvm │ │ └─pve-data 253:6 0 17.6G 0 lvm │ └─pve-data_tdata 253:5 0 17.6G 0 lvm │ └─pve-data 253:6 0 17.6G 0 lvm └─sda4 8:4 0 871G 0 part └─ceph--9ff535a4--bf8c--4052--99ef--c396ede7c8bb-osd--block--a83fd1e0--5c7b--4839--a40d--a5f553621201 253:3 0 871G 0 lvm sdb 8:16 0 931G 0 disk └─ceph--b7073224--e6d4--45a4--bbd7--ab4defe512e8-osd--block--4eaf61d2--8938--4131--a2a1--81db59fd5298 253:0 0 931G 0 lvm sdc 8:32 0 931G 0 disk sdd 8:48 0 465.3G 0 disk sde 8:64 0 111.3G 0 disk sdf 8:80 0 4T 0 disk - # this is a LUN on the NetApp sdg 8:96 0 4T 0 disk - # this is a LUN on the NetApp

 

[bbgeek17]

I have also set up an LVM using the iSCSI connection.

the screenshots and outputs as provided do not show that you setup LVM using iSCSI as it relates to PVE. What exactly did you mean here?

BUT, the LVM shows up in the PM node with a "?" and showing as not active.

what is the output of "pvesm status" ?

I can use the LUN directly and create a VM using it

As @Dunuin asked originally - what exactly do you mean here? Direct lun?

sdf 8:80 0 4T 0 disk - # this is a LUN on the NetApp sdg 8:96 0 4T 0 disk - # this is a LUN on the NetApp

are there two luns on the netapp, or is this the same lun seen twice?

If its the same lun : install multipath, configure it, create PV/VG on the resulting mpath device, add it to PVE
If its two luns: create PV/VG group as desired using direct disks, ie /dev/sdx, add it to PVE

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[MoniPM]

the screenshots and outputs as provided do not show that you setup LVM using iSCSI as it relates to PVE. What exactly did you mean here?

I created the LVM Aat the DataCenter level.

what is the output of "pvesm status" ?

Name Type Status Total Used Available %
NetApp iscsi active 0 0 0 0.00%
local dir active 25558020 4802588 19431812 18.79%
local-lvm lvmthin active 18411520 0 18411520 0.00%
netapp lvm inactive 0 0 0 0.00%

As @Dunuin asked originally - what exactly do you mean here? Direct lun?

Yes, direct LUN worked so I assumed that PM can talk to the NetApp.

are there two luns on the netapp, or is this the same lun seen twice?

There are two but one is 1TB and the other is 4TB so it looks like the 4TB is listed twice.

If its the same lun : install multipath, configure it, create PV/VG on the resulting mpath device, add it to PVE
If its two luns: create PV/VG group as desired using direct disks, ie /dev/sdx, add it to PVE

Ok, thanks

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[bbgeek17]

netapp lvm inactive 0 0 0 0.00%

You did create a configuration storage object in PVE, but you didnt point it to a valid LVM group. So the object is inactive.

Here is a guide to creating an LVM storage https://pve.proxmox.com/wiki/Logical_Volume_Manager_(LVM) but your steps may differ a bit

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox