[SOLVED] WARNING: PV 0tO1MK-jzPE-WKj4-RBi6-Ev9t-4Yvf-jXgNGF on /dev/sde was already found on /dev/mapper/san3

aasami

Renowned Member
Mar 9, 2016
55
9
73
24
Hi, I have a PVE cluster where all nodes are connected to several SANs using multipath.
However one PV in LVM complains, that it is using one of the paths instead of multipath device:
Code:
[node5@15:35 ~]# pvdisplay
  WARNING: PV 0tO1MK-jzPE-WKj4-RBi6-Ev9t-4Yvf-jXgNGF on /dev/sde was already found on /dev/mapper/san3.
  WARNING: PV 0tO1MK-jzPE-WKj4-RBi6-Ev9t-4Yvf-jXgNGF prefers device /dev/sde because device is used by LV.
  --- Physical volume ---
  PV Name               /dev/sde
  VG Name               vg03
  PV Size               1,02 TiB / not usable 3,62 MiB
  Allocatable           yes
  PE Size               4,00 MiB
  Total PE              267097
  Free PE               93273
  Allocated PE          173824
  PV UUID               0tO1MK-jzPE-WKj4-RBi6-Ev9t-4Yvf-jXgNGF
  
  --- Physical volume ---
  PV Name               /dev/mapper/san1
  VG Name               vg01
  PV Size               1,00 TiB / not usable 16,00 MiB
  Allocatable           yes
  PE Size               4,00 MiB
  Total PE              262140
  Free PE               97276
  Allocated PE          164864
  PV UUID               bvQWLj-PhGs-FcOz-h9hf-a2vZ-MxGr-2YVU6x
  
  --- Physical volume ---
  PV Name               /dev/mapper/san2
  VG Name               vg02
  PV Size               931,32 GiB / not usable 4,00 MiB
  Allocatable           yes
  PE Size               4,00 MiB
  Total PE              238417
  Free PE               238417
  Allocated PE          0
  PV UUID               d1j1ny-QhSL-IeYr-C1bf-Pv0f-oBFg-GZI82X
  
  --- Physical volume ---
  PV Name               /dev/sda3
  VG Name               pve
  PV Size               279,11 GiB / not usable 4,28 MiB
  Allocatable           yes
  PE Size               4,00 MiB
  Total PE              71452
  Free PE               4040
  Allocated PE          67412
  PV UUID               21mnuf-cCf2-XMHG-h5K9-Qep8-Ao4Y-Z6aSdP
  
[node5@10:38 ~]#
All VMs were moved out of node5.
All other nodes have the same LUNs from the same SANs configured identically.
Is it save to deactivate vg03 and how do I force LVM to use multipath device /dev/mapper/san3 to have redundancy back?
Please advice.

Here are some more details about connected LUNs on node5:
Code:
[node5@10:50 ~]# lsblk
NAME                        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                           8:0    0 279,4G  0 disk
├─sda1                        8:1    0     1M  0 part
├─sda2                        8:2    0   256M  0 part
└─sda3                        8:3    0 279,1G  0 part
  ├─pve-swap                253:24   0    32G  0 lvm   [SWAP]
  ├─pve-root                253:25   0    20G  0 lvm   /
  ├─pve-data_tmeta          253:28   0   108M  0 lvm   
  │ └─pve-data              253:30   0 211,1G  0 lvm   
  └─pve-data_tdata          253:29   0 211,1G  0 lvm   
    └─pve-data              253:30   0 211,1G  0 lvm   
sdb                           8:16   0     1T  0 disk
└─san3                      253:26   0     1T  0 mpath
sdc                           8:32   0   500G  0 disk
└─san4                      253:27   0   500G  0 mpath
sdd                           8:48   0   500G  0 disk
└─san4                      253:27   0   500G  0 mpath
sde                           8:64   0     1T  0 disk
├─vg03-vm--108--disk--2     253:0    0    20G  0 lvm   
├─and many other virtual disk volumes   
└─san3                      253:26   0     1T  0 mpath
sdj                           8:144  0 931,3G  0 disk
└─san2                      253:31   0 931,3G  0 mpath
sdk                           8:160  0 931,3G  0 disk
└─san2                      253:31   0 931,3G  0 mpath
sdl                           8:176  0     1T  0 disk
└─san1                      253:32   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1  253:33   0     5G  0 lvm   
  └─and many other virtual disk volumes   
sdm                           8:192  0     1T  0 disk
└─san1                      253:32   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1  253:33   0     5G  0 lvm   
  └─and many other virtual disk volumes   
sdn                           8:208  0     1T  0 disk
└─san1                      253:32   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1  253:33   0     5G  0 lvm   
  └─and many other virtual disk volumes   
sdo                           8:224  0     1T  0 disk
└─san1                      253:32   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1  253:33   0     5G  0 lvm   
  └─and many other virtual disk volumes   
sr0                          11:0    1  1024M  0 rom   
[node5@10:50 ~]#
For reference, here is how it looks like one node8:
Code:
[node8@12:07 ~]# lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                            8:0    0 279,4G  0 disk
├─sda1                         8:1    0     1M  0 part
├─sda2                         8:2    0   256M  0 part
└─sda3                         8:3    0 279,1G  0 part
  ├─pve-swap                 253:0    0    40G  0 lvm   [SWAP]
  ├─pve-root                 253:1    0    20G  0 lvm   /
  ├─pve-data_tmeta           253:4    0     2G  0 lvm   
  │ └─pve-data               253:6    0 199,1G  0 lvm   
  └─pve-data_tdata           253:5    0 199,1G  0 lvm   
    └─pve-data               253:6    0 199,1G  0 lvm   
sdb                            8:16   0     1T  0 disk
└─san3                       253:2    0     1T  0 mpath
  ├─vg03-vm--108--disk--2    253:7    0    20G  0 lvm   
  └─and many other virtual disk volumes
sdc                            8:32   0     1T  0 disk
└─san3                       253:2    0     1T  0 mpath
  ├─vg03-vm--108--disk--2    253:7    0    20G  0 lvm   
  └─and many other virtual disk volumes
sdd                            8:48   0   500G  0 disk
└─san4                       253:3    0   500G  0 mpath
sde                            8:64   0   500G  0 disk
└─san4                       253:3    0   500G  0 mpath
sdf                            8:80   0     1T  0 disk
└─san3                       253:2    0     1T  0 mpath
  ├─vg03-vm--108--disk--2    253:7    0    20G  0 lvm   
  └─and many other virtual disk volumes
sdg                            8:96   0     1T  0 disk
└─san3                       253:2    0     1T  0 mpath
  ├─vg03-vm--108--disk--2    253:7    0    20G  0 lvm   
  └─and many other virtual disk volumes
sdh                            8:112  0   500G  0 disk
└─san4                       253:3    0   500G  0 mpath
sdi                            8:128  0   500G  0 disk
└─san4                       253:3    0   500G  0 mpath
sdj                            8:144  0 931,3G  0 disk
└─san2                       253:20   0 931,3G  0 mpath
sdk                            8:160  0 931,3G  0 disk
└─san2                       253:20   0 931,3G  0 mpath
sdl                            8:176  0     1T  0 disk
└─san1                       253:21   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1   253:22   0     5G  0 lvm   
  └─and many other virtual disk volumes
sdm                            8:192  0     1T  0 disk
└─san1                       253:21   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1   253:22   0     5G  0 lvm
  └─and many other virtual disk volumes
sdn                            8:208  0     1T  0 disk
└─san1                       253:21   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1   253:22   0     5G  0 lvm   
  └─and many other virtual disk volumes
sdo                            8:224  0     1T  0 disk
└─san1                       253:21   0     1T  0 mpath
  ├─vg01-vm--7357--disk--1   253:22   0     5G  0 lvm   
  └─and many other virtual disk volumes
sr0                           11:0    1  1024M  0 rom   
[node8@12:07 ~]#
Thank you beforehand.
 
hmm - You maybe should exclude the paths from the lvm-scanning by adding them to the global_filter in '/etc/lvm/lvm.conf'
I hope this helps!
 
  • Like
Reactions: aasami
Glad that the config-change worked!
Thanks for reporting back!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!