New Installation on 2 HP-Proliant DL380G7 - MSAP2000G3

jerry said:
I think you should do following steps:
In the console of PREF33-S-PMOX1 or PREF33-S-PMOX2:
pvcreate /dev/mapper/3600c0ff0001ae601d4d8b05a01000000
vgcreate vmstore1 /dev/mapper/3600c0ff0001ae601d4d8b05a01000000

In the PVE GUI:
Datacenter --> Storage --> Add --> LVM
Id: your friendly name eg. vmstore1
Base storage: Existing volume groups
Volume group: you should see and choose vmstore1
Enable: checkbox - selected by default
Shared: checkbox - you should select it, because your VG will be used by two hosts
Now you will see vmstore1 below local and local-lvm and you can use it as virtual machine disk(s).

According to this document there is no need to use HP Device-Mapper Multipath Enablement Kit (and create /etc/multipath.conf file) beceause of native Linux multipath is now integrated and distributed for most of the latest Linux Releases.
Click to expand...

Oh thank you, i try this
 
Hi,

Jerry : i don't have /dev/mapper/3600c0ff0001ae601d4d8b05a01000000 entry to make pvcreate

i try to made 2 partition on Fiber storage (8Tb)
fdisk /dev/sdb


Disk /dev/sdb: 7.6 TiB, 8393144926208 bytes, 16392861184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 53499EC8-91CE-4EF1-9243-904B5D2A42B6

Device Start End Sectors Size Type
/dev/sdb1 2048 8120174591 8120172544 3.8T Linux LVM
/dev/sdb2 8120174592 16392861150 8272686559 3.9T Linux LVM


Disk /dev/sdc: 7.6 TiB, 8393144926208 bytes, 16392861184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 53499EC8-91CE-4EF1-9243-904B5D2A42B6


Disk /dev/sdd: 7.6 TiB, 8393144926208 bytes, 16392861184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 53499EC8-91CE-4EF1-9243-904B5D2A42B6


Disk /dev/sde: 7.6 TiB, 8393144926208 bytes, 16392861184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 53499EC8-91CE-4EF1-9243-904B5D2A42B6


Disk /dev/mapper/mpathb: 7.6 TiB, 8393144926208 bytes, 16392861184 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 1048576 bytes
Disklabel type: gpt
Disk identifier: 53499EC8-91CE-4EF1-9243-904B5D2A42B6

Device Start End Sectors Size Type
/dev/mapper/mpathb-part1 2048 8120174591 8120172544 3.8T Linux LVM
/dev/mapper/mpathb-part2 8120174592 16392861150 8272686559 3.9T Linux LVM

impossible to do pvcreate /dev/sdb1 or /dev/sdb2
i try to do pvcreate /dev/mapper/mpathb-part1 or part2

same error: device not found.

lsblk have a good informations : (2 partition exists)

root@PREF33-S-PMOX1:/dev/mapper# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 136.7G 0 disk
├─sda1 8:1 0 1M 0 part
├─sda2 8:2 0 256M 0 part
└─sda3 8:3 0 136.5G 0 part
├─pve-swap 253:0 0 8G 0 lvm [SWAP]
├─pve-root 253:1 0 34G 0 lvm /
├─pve-data_tmeta 253:3 0 80M 0 lvm
│ └─pve-data 253:5 0 78.5G 0 lvm
└─pve-data_tdata 253:4 0 78.5G 0 lvm
└─pve-data 253:5 0 78.5G 0 lvm
sdb 8:16 0 7.6T 0 disk
├─sdb1 8:17 0 3.8T 0 part
├─sdb2 8:18 0 3.9T 0 part
└─mpathb 253:2 0 7.6T 0 mpath
├─mpathb-part1 253:6 0 3.8T 0 part
└─mpathb-part2 253:7 0 3.9T 0 part
sdc 8:32 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
├─mpathb-part1 253:6 0 3.8T 0 part
└─mpathb-part2 253:7 0 3.9T 0 part
sdd 8:48 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
├─mpathb-part1 253:6 0 3.8T 0 part
└─mpathb-part2 253:7 0 3.9T 0 part
sde 8:64 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
├─mpathb-part1 253:6 0 3.8T 0 part
└─mpathb-part2 253:7 0 3.9T 0 part
sr0 11:0 1 1024M 0 rom

why i can't do pvcreate (device is not found) ?
 
Last edited:
Zaqen said:
Hi,

Jerry : i don't have /dev/mapper/3600c0ff0001ae601d4d8b05a01000000 entry to make pvcreate
Click to expand...
It seems that you've got friendly names because you're using multipath configuration files: /etc/multipath.conf or/and /etc/multipath/bindings

Zaqen said:
why i can't do pvcreate (device is not found) ?
Click to expand...

I'm not sure but it can be some issues creating LVM on GPT disk or some issues with LVM filtering.
You could try to remove partitions from disk and create PV on whole disk:
pvcreate /dev/mapper/mpathb
 
Last edited:
  • Like
Reactions: menk
yes i have /etc/multipath/bindings file , and /etc/multipath/wwids file.

file contains this :

root@PREF33-S-PMOX1:/etc/multipath# cat wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/3600c0ff0001ae601d4d8b05a01000000/
root@PREF33-S-PMOX1:/etc/multipath# cat bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
mpatha 3600508b1001ca15b462cb999f65494ed
root@PREF33-S-PMOX1:/etc/multipath#

They are a mistake no ?

mpatha 3600508b1001ca15b462cb999f65494ed in /etc/multipath/bindings will must contain same id of wwids file ?

syntax is it ok in /etc/multipath/wwids file ?
 
jerry said:
I'm not sure but it can be some issues creating LVM on GPT disk or some issues with LVM filtering.
You could try to remove partitions from disk and create PV on whole disk:
pvcreate /dev/mapper/mpathb
Click to expand...

yesterday i try this, impossible, same error.
 
i found an error in kernel.log


Apr 26 16:40:00 PREF33-S-PMOX2 kernel: [84618.554383] device-mapper: table: 253:6: multipath: error getting device
Apr 26 16:40:00 PREF33-S-PMOX2 kernel: [84618.554489] device-mapper: ioctl: error adding target to table
Apr 26 16:41:51 PREF33-S-PMOX2 pvedaemon[23485]: <root@pam> successful auth for user 'root@pam'
Apr 26 16:51:29 PREF33-S-PMOX2 kernel: [85307.776092] Emulex LightPulse Fibre Channel SCSI driver 11.2.0.2
Apr 26 16:51:29 PREF33-S-PMOX2 kernel: [85307.776093] Copyright(c) 2004-2016 Emulex. All rights reserved.
Apr 26 16:56:51 PREF33-S-PMOX2 pvedaemon[23485]: <root@pam> successful auth for user 'root@pam'
Apr 26 17:11:51 PREF33-S-PMOX2 pvedaemon[31291]: <root@pam> successful auth for user 'root@pam'
Apr 26 17:26:51 PREF33-S-PMOX2 pvedaemon[31291]: <root@pam> successful auth for user 'root@pam'
Apr 26 17:41:51 PREF33-S-PMOX2 pvedaemon[23485]: <root@pam> successful auth for user 'root@pam'
Apr 26 17:42:51 PREF33-S-PMOX2 kernel: [88390.048516] device-mapper: table: 253:6: multipath: error getting device
Apr 26 17:42:51 PREF33-S-PMOX2 kernel: [88390.048628] device-mapper: ioctl: error adding target to table

what missing ?
 
Zaqen said:
mpatha 3600508b1001ca15b462cb999f65494ed in /etc/multipath/bindings will must contain same id of wwids file ?

syntax is it ok in /etc/multipath/wwids file ?
Click to expand...
Files in /etc/multipath/ directory must have the same wwids inside!
Syntax in /etc/multipath/wwids is ok, but you should not to edit this file.
 
Zaqen said:
yesterday i try this, impossible, same error.
Click to expand...
Maybe, the problem is that you're using wwids in binding file that are not present in the system.
Remove any records from /etc/multipath/bindings file and remove /etc/multipath.conf file.
Reload multipath daemon using multipath -r command. In the output of multipath -ll command you will see wwids that you have to use in pvcreate command.
 
Jerry, thank you to your response.I'm desperate and I think we'll buy a vmware license.

I try to delete multipath.conf, delete entry in /etc/bindings file... restart daemon... on PMOX1 and PMOX2

in kernel log i have the same error :

Apr 27 09:43:04 PREF33-S-PMOX1 kernel: [ 15.751630] audit: type=1400 audit(1524814984.782:8): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="lxc-container-default" pid=1353 comm="apparmor_parser"
Apr 27 09:43:04 PREF33-S-PMOX1 kernel: [ 15.751633] audit: type=1400 audit(1524814984.782:9): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="lxc-container-default-cgns" pid=1353 comm="apparmor_parser"
Apr 27 09:43:04 PREF33-S-PMOX1 kernel: [ 15.751635] audit: type=1400 audit(1524814984.782:10): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="lxc-container-default-with-mounting" pid=1353 comm="apparmor_parser"
Apr 27 09:43:04 PREF33-S-PMOX1 kernel: [ 15.751636] audit: type=1400 audit(1524814984.782:11): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="lxc-container-default-with-nesting" pid=1353 comm="apparmor_parser"
Apr 27 09:43:06 PREF33-S-PMOX1 kernel: [ 17.379652] e1000e: ens3f0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Apr 27 09:43:06 PREF33-S-PMOX1 kernel: [ 17.379807] vmbr0: port 1(ens3f0) entered blocking state
Apr 27 09:43:06 PREF33-S-PMOX1 kernel: [ 17.379809] vmbr0: port 1(ens3f0) entered forwarding state
Apr 27 09:43:06 PREF33-S-PMOX1 kernel: [ 17.379882] IPv6: ADDRCONF(NETDEV_CHANGE): vmbr0: link becomes ready
Apr 27 09:43:08 PREF33-S-PMOX1 kernel: [ 19.000727] ip6_tables: (C) 2000-2006 Netfilter Core Team
Apr 27 09:43:08 PREF33-S-PMOX1 kernel: [ 19.062409] ip_set: protocol 6
Apr 27 09:43:12 PREF33-S-PMOX1 pve-manager[1601]: <root@pam> starting task UPID:PREF33-S-PMOX1:00000649:00000926:5AE2D490:startall::root@pam:
Apr 27 09:43:12 PREF33-S-PMOX1 pve-manager[1601]: <root@pam> end task UPID:PREF33-S-PMOX1:00000649:00000926:5AE2D490:startall::root@pam: OK
Apr 27 09:48:32 PREF33-S-PMOX1 pvedaemon[1583]: <root@pam> successful auth for user 'root@pam'
Apr 27 10:01:54 PREF33-S-PMOX1 kernel: [ 1145.775402] device-mapper: multipath service-time: version 0.3.0 loaded
Apr 27 10:01:54 PREF33-S-PMOX1 kernel: [ 1145.777708] device-mapper: table: 253:6: multipath: error getting device
Apr 27 10:01:54 PREF33-S-PMOX1 kernel: [ 1145.777820] device-mapper: ioctl: error adding target to table
Apr 27 10:03:32 PREF33-S-PMOX1 pvedaemon[1582]: <root@pam> successful auth for user 'root@pam'
Apr 27 10:09:28 PREF33-S-PMOX1 kernel: [ 1599.724612] device-mapper: table: 253:6: multipath: error getting device
Apr 27 10:09:28 PREF33-S-PMOX1 kernel: [ 1599.724716] device-mapper: ioctl: error adding target to table

GRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
 
Question, pvcreate must do with root account ?

I try to create with uuid :

root@PREF33-S-PMOX2:/dev/mapper# pvcreate /dev/mapper/3600c0ff0001ae601d4d8b05a01000000
Device /dev/mapper/3600c0ff0001ae601d4d8b05a01000000 not found (or ignored by filtering).
root@PREF33-S-PMOX2:/dev/mapper# ls
3600c0ff0001ae601d4d8b05a01000000 control pve-data pve-data_tdata pve-data_tmeta pve-root pve-swap
root@PREF33-S-PMOX2:/dev/mapper# ls -lai
total 0
1179 drwxr-xr-x 2 root root 180 Apr 27 10:02 .
1025 drwxr-xr-x 21 root root 4520 Apr 27 10:02 ..
129172 lrwxrwxrwx 1 root root 7 Apr 27 10:02 3600c0ff0001ae601d4d8b05a01000000 -> ../dm-2
1180 crw------- 1 root root 10, 236 Apr 27 09:43 control
21651 lrwxrwxrwx 1 root root 7 Apr 27 09:43 pve-data -> ../dm-5
20588 lrwxrwxrwx 1 root root 7 Apr 27 09:43 pve-data_tdata -> ../dm-4
23701 lrwxrwxrwx 1 root root 7 Apr 27 09:43 pve-data_tmeta -> ../dm-3
19470 lrwxrwxrwx 1 root root 7 Apr 27 09:43 pve-root -> ../dm-1
12457 lrwxrwxrwx 1 root root 7 Apr 27 09:43 pve-swap -> ../dm-0
root@PREF33-S-PMOX2:/dev/mapper#
 
when i do :

cd /dev/map (and press [TAB]) completion not complete /dev/mapper
is normal ?
 
OK

root@PREF33-S-PMOX1:~# multipath -ll
3600c0ff0001ae601d4d8b05a01000000 dm-2 HP,P2000 G3 FC
size=7.6T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 4:0:1:1 sde 8:64 active ready running
| `- 2:0:1:1 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 4:0:0:1 sdd 8:48 active ready running
`- 2:0:0:1 sdb 8:16 active ready running


root@PREF33-S-PMOX2:~#
3600c0ff0001ae601d4d8b05a01000000 dm-2 HP,P2000 G3 FC
size=7.6T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 4:0:1:1 sde 8:64 active ready running
| `- 2:0:1:1 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 4:0:0:1 sdd 8:48 active ready running
`- 2:0:0:1 sdb 8:16 active ready running
root@PREF33-S-PMOX2:/dev#
 
Last edited:
Hi,
I download a last version ISO, i do new installation on PMOX1 and PMOX2, installation finish and i have web interface running
on PMOX1 and PMOX2.

Installation is new with last version, 5.1.3 kernel 4.13.13-2-pve, on two servers proliant.

Système pmox1 & pmox2 not update (no internet connexion, i must do repository ton install and update more tools)
I do this in first time

Fc link is ok, pmox1 et 2 seeing /dev/sdb /dev/sdc /dev/sdd /dev/sde

What must à i do now ?
 
  • Like
Reactions: menk
Zaqen said:
Hi,
I download a last version ISO, i do new installation on PMOX1 and PMOX2, installation finish and i have web interface running
on PMOX1 and PMOX2.

Installation is new with last version, 5.1.3 kernel 4.13.13-2-pve, on two servers proliant.

Système pmox1 & pmox2 not update (no internet connexion, i must do repository ton install and update more tools)
I do this in first time

Fc link is ok, pmox1 et 2 seeing /dev/sdb /dev/sdc /dev/sdd /dev/sde

What must à i do now ?
Click to expand...
you have to configure multipath, and then create a lvm on this. ;)
 
Zaqen said:
when i do :

cd /dev/map (and press [TAB]) completion not complete /dev/mapper
is normal ?
Click to expand...

Yes, it's (sometimes) normal behaviour.

Zaqen said:
OK

root@PREF33-S-PMOX1:~# multipath -ll
3600c0ff0001ae601d4d8b05a01000000 dm-2 HP,P2000 G3 FC
size=7.6T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 4:0:1:1 sde 8:64 active ready running
| `- 2:0:1:1 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 4:0:0:1 sdd 8:48 active ready running
`- 2:0:0:1 sdb 8:16 active ready running


root@PREF33-S-PMOX2:~#
3600c0ff0001ae601d4d8b05a01000000 dm-2 HP,P2000 G3 FC
size=7.6T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 4:0:1:1 sde 8:64 active ready running
| `- 2:0:1:1 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 4:0:0:1 sdd 8:48 active ready running
`- 2:0:0:1 sdb 8:16 active ready running
root@PREF33-S-PMOX2:/dev#
Click to expand...

Zaqen, could you check output of this command:
pvcreate -vvvv /dev/mapper/3600c0ff0001ae601d4d8b05a01000000

If you will find something like "Skipping: Unrecognised LVM device type 253" you should add found magic number to /etc/lvm/lvm.conf (in the devices section):
types = [ "bcache", 253 ]
Looking at your kernel.log I suppose it is 253.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom