New Installation on 2 HP-Proliant DL380G7 - MSAP2000G3

Z

Zaqen

Active Member
Mar 20, 2018
58
3
28
54
Hi,

I have installing Proxmox V5.0 on 2 HP proliant DL380 and i want use SAN storage (approximately 45TB)

Proxmox 1
Proxmox 2

are 2 server with proxmox (i have create cluster, add 2nd proxmo2 on proxmo1). the 2 server have 4 interface fiber channel to connecting on san.

I have create one lun of 7,3 TB to proxmox.

My 2 servers can see 4 Lun, in reality they are juste one lun, proxmox1 server and proxmo2 server, seeing 4 lun (same number and same capacity) i think because they are 4 fiber channel per server.

So, i think it's ok to work, but i don't no what must i do now to use lun and create storage.

When i m on server promox 1 or proxmox 2 i see lun on command line :

root@proxmox1# fdisk /dev/MyLun is possible, i can create partition, change type etc....


Can you help me ?
 
thank's to your response, but not information to pve 5 configuration device on it. on pve 5, i need to create /etc/multipath.conf ?

multipath -ll is ok i see 4 disk who is on my lun

:~# multipath -ll
3600c0ff0001ae601d4d8b05a01000000 dm-2 HP,P2000 G3 FC
size=7.6T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 4:0:1:1 sde 8:64 active ready running
| `- 2:0:1:1 sdc 8:32 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 4:0:0:1 sdd 8:48 active ready running
`- 2:0:0:1 sdb 8:16 active ready running

In proxmox what must i do to use it ? I don't understand
 
read the section "multipath configuration"; you have to configuge multipath daemon for use round robin.
so edit /etc/multipath.conf for your configuration.
this is example of my old configuration in PVE 4.x with 2 node and a SAS San Lenovo, but i don't think there is difference to pve 5.x:

defaults {
polling_interval 2
path_selector "round-robin 0"
path_grouping_policy multibus
uid_attribute ID_SERIAL
rr_min_io 100
failback immediate
no_path_retry queue
user_friendly_names yes
}
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^(td|hd)[a-z]"
devnode "^dcssblk[0-9]*"
devnode "^cciss!c[0-9]d[0-9]*"
device {
vendor "DGC"
product "LUNZ"
}
device {
vendor "EMC"
product "LUNZ"
}
device {
vendor "IBM"
product "Universal Xport"
}
device {
vendor "IBM"
product "S/390.*"
}
device {
vendor "DELL"
product "Universal Xport"
}
device {
vendor "SGI"
product "Universal Xport"
}
device {
vendor "STK"
product "Universal Xport"
}
device {
vendor "SUN"
product "Universal Xport"
}
device {
vendor "(NETAPP|LSI|ENGENIO)"
product "Universal Xport"
}
}
blacklist_exceptions {
wwid "36005076300808d10e000000000000000"
}
multipaths {
multipath {
wwid "36005076300808d10e000000000000000"
alias mpathb
}
 
Last edited:
Thank'you,

I do one file who copy it on 2 nodes (proxmox1 and 2) :

/etc/multipath.conf

defaults {
polling_interval 2
path_selector "round-robin 0"
path_grouping_policy multibus
getuid_callout "/lib/udev/scsi_id -g -u -d /dev/%n"
rr_min_io 100
failback immediate
no_path_retry queue
}
blacklist {
wwid .*
}

blacklist_exceptions {
wwid 3600c0ff0001ae601d4d8b05a01000000

}

devices {
device {
vendor "HP"
product "MSAP2000G3"
path_grouping_policy group_by_prio
prio rdac
path_checker rdac
path_selector "round-robin 0"
hardware_handler "1 rdac"
failback immediate
features "2 pg_init_retries 50"
no_path_retry 30
rr_min_io 100
}
}

multipaths {
multipath {
wwid 3600c0ff0001ae601d4d8b05a01000000
alias HPMSAP2000G3
}

}

In log no error when restart service (systemd start multipath.service) :


Mar 21 17:46:56 PREF33-S-PMOX1 systemd[1]: Starting Device-Mapper Multipath Device Controller...
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: path checkers start up
Mar 21 17:46:56 PREF33-S-PMOX1 systemd[1]: Started Device-Mapper Multipath Device Controller.
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: sda: using deprecated getuid callout
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: sdd: using deprecated getuid callout
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: sde: using deprecated getuid callout
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: sdb: using deprecated getuid callout
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: sdc: using deprecated getuid callout
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: HPMSAP2000G3: load table [0 16392861184 multipath 2 queue_if_no_path retain_attached_hw_handler 0 2 1 round-robin 0 2 1 8:64 1 8:32 1 round-robin 0 2 1 8:48 1 8:16 1]
Mar 21 17:46:56 PREF33-S-PMOX1 multipathd[3537]: HPMSAP2000G3: event checker started
Mar 21 17:47:00 PREF33-S-PMOX1 systemd[1]: Starting Proxmox VE replication runner...
Mar 21 17:47:01 PREF33-S-PMOX1 systemd[1]: Started Proxmox VE replication runner.

I have no error in log, but what must i do now ?
 
now you must create lvm on top of the lun so you can use it as shared storage on the two node......
you can't use lun directly on a cluster, it isn't cluster aware.....
 
OK but where i do this, i don't understand where i can make this ? is most to make lvm thin or lvm ?
 
I don't understand howto add Lun to create Storage.

All pictures are here : https://photos.app.goo.gl/VPSwnx1tweCNtKVo1

PMOX1 & PMOX2 seed the LUN (i have create VDISK on HP P2000 G3 FC of 8tb) it's ok

I configure multipath (configuration 1 & 2 on pictures), processus run (on pmox 1 et pmox 2) ,

multipath -ll return good status (i think)


Now, i try to add iscsi disk on proxmox gui interface, but i don't know what must i do.

San HP 2000 G3 FC, have 2 controllers (10.33.71.245 / 10.33.71.246)

multpath v3 return other information (pictures), i don't see error, what do you think ?

root@PREF33-S-PMOX1:~# multipath -v3
Apr 24 14:40:34 | libdevmapper version 1.02.137 (2016-11-30)
Apr 24 14:40:34 | DM multipath kernel driver v1.12.0
Apr 24 14:40:34 | loading //lib/multipath/libchecktur.so checker
Apr 24 14:40:34 | loading //lib/multipath/libprioconst.so prioritizer
Apr 24 14:40:34 | sda: udev property ID_WWN whitelisted
Apr 24 14:40:34 | sda: mask = 0x1f
Apr 24 14:40:34 | sda: dev_t = 8:0
Apr 24 14:40:34 | sda: size = 286677120
Apr 24 14:40:34 | sda: vendor = HP
Apr 24 14:40:34 | sda: product = LOGICAL VOLUME
Apr 24 14:40:34 | sda: rev = 3.52
Apr 24 14:40:34 | sda: h:b:t:l = 3:1:0:0
Apr 24 14:40:34 | sda: tgt_node_name =
Apr 24 14:40:34 | sda: path state = running
Apr 24 14:40:34 | sda: 17844 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:40:34 | sda: serial = 5001438009E83B30
Apr 24 14:40:34 | sda: get_state
Apr 24 14:40:34 | sda: path_checker = tur (internal default)
Apr 24 14:40:34 | sda: checker timeout = 30 ms (internal default)
Apr 24 14:40:34 | sda: state = up
Apr 24 14:40:34 | sda: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:40:34 | sda: using deprecated getuid callout
Apr 24 14:40:34 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sda
Apr 24 14:40:34 | sda: uid = 3600508b1001ca15b462cb999f65494ed (callout)
Apr 24 14:40:34 | sda: detect_prio = yes (config file default)
Apr 24 14:40:34 | sda: prio = const (internal default)
Apr 24 14:40:34 | sda: prio args = "" (internal default)
Apr 24 14:40:34 | sda: const prio = 1
Apr 24 14:40:34 | sdd: udev property ID_WWN whitelisted
Apr 24 14:40:34 | sdd: mask = 0x1f
Apr 24 14:40:34 | sdd: dev_t = 8:48
Apr 24 14:40:34 | sdd: size = 16392861184
Apr 24 14:40:34 | sdd: vendor = HP
Apr 24 14:40:34 | sdd: product = P2000 G3 FC
Apr 24 14:40:34 | sdd: rev = T252
Apr 24 14:40:34 | sdd: h:b:t:l = 4:0:0:1
Apr 24 14:40:34 | SCSI target 4:0:0 -> FC rport 4:0-0
Apr 24 14:40:34 | sdd: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:40:34 | sdd: path state = running
Apr 24 14:40:34 | sdd: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:40:34 | sdd: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:40:34 | sdd: get_state
Apr 24 14:40:34 | sdd: path_checker = tur (internal default)
Apr 24 14:40:34 | sdd: checker timeout = 30 ms (internal default)
Apr 24 14:40:34 | sdd: state = up
Apr 24 14:40:34 | sdd: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:40:34 | sdd: using deprecated getuid callout
Apr 24 14:40:34 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sdd
Apr 24 14:40:34 | sdd: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:40:34 | sdd: detect_prio = yes (config file default)
Apr 24 14:40:34 | 4:0:0:1: attribute access_state not found in sysfs
Apr 24 14:40:34 | loading //lib/multipath/libprioalua.so prioritizer
Apr 24 14:40:34 | sdd: prio = alua (detected setting)
Apr 24 14:40:34 | sdd: prio args = "" (detected setting)
Apr 24 14:40:34 | sdd: reported target port group is 1
Apr 24 14:40:34 | sdd: aas = 01 [active/non-optimized]
Apr 24 14:40:34 | sdd: alua prio = 10
Apr 24 14:40:34 | sde: udev property ID_WWN whitelisted
Apr 24 14:40:34 | sde: mask = 0x1f
Apr 24 14:40:34 | sde: dev_t = 8:64
Apr 24 14:40:34 | sde: size = 16392861184
Apr 24 14:40:34 | sde: vendor = HP
Apr 24 14:40:34 | sde: product = P2000 G3 FC
Apr 24 14:40:34 | sde: rev = T252
Apr 24 14:40:34 | sde: h:b:t:l = 4:0:1:1
Apr 24 14:40:34 | SCSI target 4:0:1 -> FC rport 4:0-1
Apr 24 14:40:34 | sde: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:40:34 | sde: path state = running
Apr 24 14:40:34 | sde: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:40:34 | sde: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:40:34 | sde: get_state
Apr 24 14:40:34 | sde: path_checker = tur (internal default)
Apr 24 14:40:34 | sde: checker timeout = 30 ms (internal default)
Apr 24 14:40:34 | sde: state = up
Apr 24 14:40:34 | sde: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:40:34 | sde: using deprecated getuid callout
Apr 24 14:40:34 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sde
Apr 24 14:40:34 | sde: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:40:34 | sde: detect_prio = yes (config file default)
Apr 24 14:40:34 | 4:0:1:1: attribute access_state not found in sysfs
Apr 24 14:40:34 | sde: prio = alua (detected setting)
Apr 24 14:40:34 | sde: prio args = "" (detected setting)
Apr 24 14:40:34 | sde: reported target port group is 0
Apr 24 14:40:34 | sde: aas = 80 [active/optimized] [preferred]
Apr 24 14:40:34 | sde: alua prio = 50
Apr 24 14:40:34 | sdb: udev property ID_WWN whitelisted
Apr 24 14:40:34 | sdb: mask = 0x1f
Apr 24 14:40:34 | sdb: dev_t = 8:16
Apr 24 14:40:34 | sdb: size = 16392861184
Apr 24 14:40:34 | sdb: vendor = HP
Apr 24 14:40:34 | sdb: product = P2000 G3 FC
Apr 24 14:40:34 | sdb: rev = T252
Apr 24 14:40:34 | sdb: h:b:t:l = 2:0:0:1
Apr 24 14:40:34 | SCSI target 2:0:0 -> FC rport 2:0-0
Apr 24 14:40:34 | sdb: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:40:34 | sdb: path state = running
Apr 24 14:40:34 | sdb: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:40:34 | sdb: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:40:34 | sdb: get_state
Apr 24 14:40:34 | sdb: path_checker = tur (internal default)
Apr 24 14:40:34 | sdb: checker timeout = 30 ms (internal default)
Apr 24 14:40:34 | sdb: state = up
Apr 24 14:40:34 | sdb: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:40:34 | sdb: using deprecated getuid callout
Apr 24 14:40:34 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sdb
Apr 24 14:40:34 | sdb: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:40:34 | sdb: detect_prio = yes (config file default)
Apr 24 14:40:34 | 2:0:0:1: attribute access_state not found in sysfs
Apr 24 14:40:34 | sdb: prio = alua (detected setting)
Apr 24 14:40:34 | sdb: prio args = "" (detected setting)
Apr 24 14:40:34 | sdb: reported target port group is 1
Apr 24 14:40:34 | sdb: aas = 01 [active/non-optimized]
Apr 24 14:40:34 | sdb: alua prio = 10
Apr 24 14:40:34 | sdc: udev property ID_WWN whitelisted
Apr 24 14:40:34 | sdc: mask = 0x1f
Apr 24 14:40:34 | sdc: dev_t = 8:32
Apr 24 14:40:34 | sdc: size = 16392861184
Apr 24 14:40:34 | sdc: vendor = HP
Apr 24 14:40:34 | sdc: product = P2000 G3 FC
Apr 24 14:40:34 | sdc: rev = T252
Apr 24 14:40:34 | sdc: h:b:t:l = 2:0:1:1
Apr 24 14:40:34 | SCSI target 2:0:1 -> FC rport 2:0-1
Apr 24 14:40:34 | sdc: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:40:34 | sdc: path state = running
Apr 24 14:40:34 | sdc: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:40:34 | sdc: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:40:34 | sdc: get_state
Apr 24 14:40:34 | sdc: path_checker = tur (internal default)
Apr 24 14:40:34 | sdc: checker timeout = 30 ms (internal default)
Apr 24 14:40:34 | sdc: state = up
Apr 24 14:40:34 | sdc: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:40:34 | sdc: using deprecated getuid callout
Apr 24 14:40:34 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sdc
Apr 24 14:40:34 | sdc: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:40:34 | sdc: detect_prio = yes (config file default)
Apr 24 14:40:34 | 2:0:1:1: attribute access_state not found in sysfs
Apr 24 14:40:34 | sdc: prio = alua (detected setting)
Apr 24 14:40:34 | sdc: prio args = "" (detected setting)
Apr 24 14:40:34 | sdc: reported target port group is 0
Apr 24 14:40:34 | sdc: aas = 80 [active/optimized] [preferred]
Apr 24 14:40:34 | sdc: alua prio = 50
Apr 24 14:40:34 | sr0: blacklisted, udev property missing
Apr 24 14:40:34 | loop0: blacklisted, udev property missing
Apr 24 14:40:34 | loop1: blacklisted, udev property missing
Apr 24 14:40:34 | loop2: blacklisted, udev property missing
Apr 24 14:40:34 | loop3: blacklisted, udev property missing
Apr 24 14:40:34 | loop4: blacklisted, udev property missing
Apr 24 14:40:34 | loop5: blacklisted, udev property missing
Apr 24 14:40:34 | loop6: blacklisted, udev property missing
Apr 24 14:40:34 | loop7: blacklisted, udev property missing
Apr 24 14:40:34 | dm-0: blacklisted, udev property missing
Apr 24 14:40:34 | dm-1: blacklisted, udev property missing
Apr 24 14:40:34 | dm-2: blacklisted, udev property missing
Apr 24 14:40:34 | dm-3: blacklisted, udev property missing
Apr 24 14:40:34 | dm-4: blacklisted, udev property missing
Apr 24 14:40:34 | dm-5: blacklisted, udev property missing
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod
3600508b1001ca15b462cb999f65494ed 3:1:0:0 sda 8:0 1 undef ready HP,LOGICA
3600c0ff0001ae601d4d8b05a01000000 4:0:0:1 sdd 8:48 10 undef ready HP,P2000
3600c0ff0001ae601d4d8b05a01000000 4:0:1:1 sde 8:64 50 undef ready HP,P2000
3600c0ff0001ae601d4d8b05a01000000 2:0:0:1 sdb 8:16 10 undef ready HP,P2000
3600c0ff0001ae601d4d8b05a01000000 2:0:1:1 sdc 8:32 50 undef ready HP,P2000
Apr 24 14:40:34 | params = 2 queue_if_no_path retain_attached_hw_handler 0 2 1 round-robin 0 2 1 8:64 1 8:32 1 round-robin 0 2 1 8:48 1 8:16 1
Apr 24 14:40:34 | status = 2 0 0 0 2 1 A 0 2 0 8:64 A 0 8:32 A 0 E 0 2 0 8:48 A 0 8:16 A 0
Apr 24 14:40:34 | HPMSAP2000G3: disassemble map [2 queue_if_no_path retain_attached_hw_handler 0 2 1 round-robin 0 2 1 8:64 1 8:32 1 round-robin 0 2 1 8:48 1 8:16 1 ]
Apr 24 14:40:34 | HPMSAP2000G3: disassemble status [2 0 0 0 2 1 A 0 2 0 8:64 A 0 8:32 A 0 E 0 2 0 8:48 A 0 8:16 A 0 ]
Apr 24 14:40:34 | sda: (HP:LOGICAL VOLUME) wwid blacklisted
Apr 24 14:40:34 | sda: orphan path, wwid blacklisted
Apr 24 14:40:34 | const prioritizer refcount 1
Apr 24 14:40:34 | tur checker refcount 5
Apr 24 14:40:34 | sdd: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:40:34 | sde: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:40:34 | sdb: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:40:34 | sdc: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:40:34 | tur checker refcount 4
Apr 24 14:40:34 | alua prioritizer refcount 4
Apr 24 14:40:34 | tur checker refcount 3
Apr 24 14:40:34 | alua prioritizer refcount 3
Apr 24 14:40:34 | tur checker refcount 2
Apr 24 14:40:34 | alua prioritizer refcount 2
Apr 24 14:40:34 | tur checker refcount 1
Apr 24 14:40:34 | alua prioritizer refcount 1
Apr 24 14:40:34 | unloading alua prioritizer
Apr 24 14:40:34 | unloading const prioritizer
Apr 24 14:40:34 | unloading tur checker
root@PREF33-S-PMOX1:~#


root@PREF33-S-PMOX2:~# multipath -v3
Apr 24 14:42:50 | libdevmapper version 1.02.137 (2016-11-30)
Apr 24 14:42:50 | DM multipath kernel driver v1.12.0
Apr 24 14:42:50 | loading //lib/multipath/libchecktur.so checker
Apr 24 14:42:50 | loading //lib/multipath/libprioconst.so prioritizer
Apr 24 14:42:50 | sda: udev property ID_WWN whitelisted
Apr 24 14:42:50 | sda: mask = 0x1f
Apr 24 14:42:50 | sda: dev_t = 8:0
Apr 24 14:42:50 | sda: size = 286677120
Apr 24 14:42:50 | sda: vendor = HP
Apr 24 14:42:50 | sda: product = LOGICAL VOLUME
Apr 24 14:42:50 | sda: rev = 3.52
Apr 24 14:42:50 | sda: h:b:t:l = 3:1:0:0
Apr 24 14:42:50 | sda: tgt_node_name =
Apr 24 14:42:50 | sda: path state = running
Apr 24 14:42:50 | sda: 17844 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:42:50 | sda: serial = 5001438009EC3AD0
Apr 24 14:42:50 | sda: get_state
Apr 24 14:42:50 | sda: path_checker = tur (internal default)
Apr 24 14:42:50 | sda: checker timeout = 30 ms (internal default)
Apr 24 14:42:50 | sda: state = up
Apr 24 14:42:50 | sda: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:42:50 | sda: using deprecated getuid callout
Apr 24 14:42:50 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sda
Apr 24 14:42:50 | sda: uid = 3600508b1001c2054c95ae4aa95dd7810 (callout)
Apr 24 14:42:50 | sda: detect_prio = yes (config file default)
Apr 24 14:42:50 | sda: prio = const (internal default)
Apr 24 14:42:50 | sda: prio args = "" (internal default)
Apr 24 14:42:50 | sda: const prio = 1
Apr 24 14:42:50 | sdd: udev property ID_WWN whitelisted
Apr 24 14:42:50 | sdd: mask = 0x1f
Apr 24 14:42:50 | sdd: dev_t = 8:48
Apr 24 14:42:50 | sdd: size = 16392861184
Apr 24 14:42:50 | sdd: vendor = HP
Apr 24 14:42:50 | sdd: product = P2000 G3 FC
Apr 24 14:42:50 | sdd: rev = T252
Apr 24 14:42:50 | sdd: h:b:t:l = 4:0:0:1
Apr 24 14:42:50 | SCSI target 4:0:0 -> FC rport 4:0-0
Apr 24 14:42:50 | sdd: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:42:50 | sdd: path state = running
Apr 24 14:42:50 | sdd: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:42:50 | sdd: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:42:50 | sdd: get_state
Apr 24 14:42:50 | sdd: path_checker = tur (internal default)
Apr 24 14:42:50 | sdd: checker timeout = 30 ms (internal default)
Apr 24 14:42:50 | sdd: state = up
Apr 24 14:42:50 | sdd: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:42:50 | sdd: using deprecated getuid callout
Apr 24 14:42:50 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sdd
Apr 24 14:42:50 | sdd: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:42:50 | sdd: detect_prio = yes (config file default)
Apr 24 14:42:50 | 4:0:0:1: attribute access_state not found in sysfs
Apr 24 14:42:50 | loading //lib/multipath/libprioalua.so prioritizer
Apr 24 14:42:50 | sdd: prio = alua (detected setting)
Apr 24 14:42:50 | sdd: prio args = "" (detected setting)
Apr 24 14:42:50 | sdd: reported target port group is 1
Apr 24 14:42:50 | sdd: aas = 01 [active/non-optimized]
Apr 24 14:42:50 | sdd: alua prio = 10
Apr 24 14:42:50 | sde: udev property ID_WWN whitelisted
Apr 24 14:42:50 | sde: mask = 0x1f
Apr 24 14:42:50 | sde: dev_t = 8:64
Apr 24 14:42:50 | sde: size = 16392861184
Apr 24 14:42:50 | sde: vendor = HP
Apr 24 14:42:50 | sde: product = P2000 G3 FC
Apr 24 14:42:50 | sde: rev = T252
Apr 24 14:42:50 | sde: h:b:t:l = 4:0:1:1
Apr 24 14:42:50 | SCSI target 4:0:1 -> FC rport 4:0-1
Apr 24 14:42:50 | sde: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:42:50 | sde: path state = running
Apr 24 14:42:50 | sde: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:42:50 | sde: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:42:50 | sde: get_state
Apr 24 14:42:50 | sde: path_checker = tur (internal default)
Apr 24 14:42:50 | sde: checker timeout = 30 ms (internal default)
Apr 24 14:42:50 | sde: state = up
Apr 24 14:42:50 | sde: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:42:50 | sde: using deprecated getuid callout
Apr 24 14:42:50 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sde
Apr 24 14:42:50 | sde: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:42:50 | sde: detect_prio = yes (config file default)
Apr 24 14:42:50 | 4:0:1:1: attribute access_state not found in sysfs
Apr 24 14:42:50 | sde: prio = alua (detected setting)
Apr 24 14:42:50 | sde: prio args = "" (detected setting)
Apr 24 14:42:50 | sde: reported target port group is 0
Apr 24 14:42:50 | sde: aas = 80 [active/optimized] [preferred]
Apr 24 14:42:50 | sde: alua prio = 50
Apr 24 14:42:50 | sdb: udev property ID_WWN whitelisted
Apr 24 14:42:50 | sdb: mask = 0x1f
Apr 24 14:42:50 | sdb: dev_t = 8:16
Apr 24 14:42:50 | sdb: size = 16392861184
Apr 24 14:42:50 | sdb: vendor = HP
Apr 24 14:42:50 | sdb: product = P2000 G3 FC
Apr 24 14:42:50 | sdb: rev = T252
Apr 24 14:42:50 | sdb: h:b:t:l = 2:0:0:1
Apr 24 14:42:50 | SCSI target 2:0:0 -> FC rport 2:0-0
Apr 24 14:42:50 | sdb: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:42:50 | sdb: path state = running
Apr 24 14:42:50 | sdb: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:42:50 | sdb: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:42:50 | sdb: get_state
Apr 24 14:42:50 | sdb: path_checker = tur (internal default)
Apr 24 14:42:50 | sdb: checker timeout = 30 ms (internal default)
Apr 24 14:42:50 | sdb: state = up
Apr 24 14:42:50 | sdb: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:42:50 | sdb: using deprecated getuid callout
Apr 24 14:42:50 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sdb
Apr 24 14:42:50 | sdb: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:42:50 | sdb: detect_prio = yes (config file default)
Apr 24 14:42:50 | 2:0:0:1: attribute access_state not found in sysfs
Apr 24 14:42:50 | sdb: prio = alua (detected setting)
Apr 24 14:42:50 | sdb: prio args = "" (detected setting)
Apr 24 14:42:50 | sdb: reported target port group is 1
Apr 24 14:42:50 | sdb: aas = 01 [active/non-optimized]
Apr 24 14:42:50 | sdb: alua prio = 10
Apr 24 14:42:50 | sdc: udev property ID_WWN whitelisted
Apr 24 14:42:50 | sdc: mask = 0x1f
Apr 24 14:42:50 | sdc: dev_t = 8:32
Apr 24 14:42:50 | sdc: size = 16392861184
Apr 24 14:42:50 | sdc: vendor = HP
Apr 24 14:42:50 | sdc: product = P2000 G3 FC
Apr 24 14:42:50 | sdc: rev = T252
Apr 24 14:42:50 | sdc: h:b:t:l = 2:0:1:1
Apr 24 14:42:50 | SCSI target 2:0:1 -> FC rport 2:0-1
Apr 24 14:42:50 | sdc: tgt_node_name = 0x208000c0ff10a940
Apr 24 14:42:50 | sdc: path state = running
Apr 24 14:42:50 | sdc: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Apr 24 14:42:50 | sdc: serial = 00c0ff1ae6010000d4d8b05a01000000
Apr 24 14:42:50 | sdc: get_state
Apr 24 14:42:50 | sdc: path_checker = tur (internal default)
Apr 24 14:42:50 | sdc: checker timeout = 30 ms (internal default)
Apr 24 14:42:50 | sdc: state = up
Apr 24 14:42:50 | sdc: getuid = "/lib/udev/scsi_id -g -u -d /dev/%n" (config file default)
Apr 24 14:42:50 | sdc: using deprecated getuid callout
Apr 24 14:42:50 | formatted callout = /lib/udev/scsi_id -g -u -d /dev/sdc
Apr 24 14:42:50 | sdc: uid = 3600c0ff0001ae601d4d8b05a01000000 (callout)
Apr 24 14:42:50 | sdc: detect_prio = yes (config file default)
Apr 24 14:42:50 | 2:0:1:1: attribute access_state not found in sysfs
Apr 24 14:42:50 | sdc: prio = alua (detected setting)
Apr 24 14:42:50 | sdc: prio args = "" (detected setting)
Apr 24 14:42:50 | sdc: reported target port group is 0
Apr 24 14:42:50 | sdc: aas = 80 [active/optimized] [preferred]
Apr 24 14:42:50 | sdc: alua prio = 50
Apr 24 14:42:50 | sr0: blacklisted, udev property missing
Apr 24 14:42:50 | loop0: blacklisted, udev property missing
Apr 24 14:42:50 | loop1: blacklisted, udev property missing
Apr 24 14:42:50 | loop2: blacklisted, udev property missing
Apr 24 14:42:50 | loop3: blacklisted, udev property missing
Apr 24 14:42:50 | loop4: blacklisted, udev property missing
Apr 24 14:42:50 | loop5: blacklisted, udev property missing
Apr 24 14:42:50 | loop6: blacklisted, udev property missing
Apr 24 14:42:50 | loop7: blacklisted, udev property missing
Apr 24 14:42:50 | dm-0: blacklisted, udev property missing
Apr 24 14:42:50 | dm-1: blacklisted, udev property missing
Apr 24 14:42:50 | dm-2: blacklisted, udev property missing
Apr 24 14:42:50 | dm-3: blacklisted, udev property missing
Apr 24 14:42:50 | dm-4: blacklisted, udev property missing
Apr 24 14:42:50 | dm-5: blacklisted, udev property missing
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod
3600508b1001c2054c95ae4aa95dd7810 3:1:0:0 sda 8:0 1 undef ready HP,LOGICA
3600c0ff0001ae601d4d8b05a01000000 4:0:0:1 sdd 8:48 10 undef ready HP,P2000
3600c0ff0001ae601d4d8b05a01000000 4:0:1:1 sde 8:64 50 undef ready HP,P2000
3600c0ff0001ae601d4d8b05a01000000 2:0:0:1 sdb 8:16 10 undef ready HP,P2000
3600c0ff0001ae601d4d8b05a01000000 2:0:1:1 sdc 8:32 50 undef ready HP,P2000
Apr 24 14:42:50 | params = 2 queue_if_no_path retain_attached_hw_handler 0 2 1 round-robin 0 2 1 8:64 1 8:32 1 round-robin 0 2 1 8:48 1 8:16 1
Apr 24 14:42:50 | status = 2 0 0 0 2 1 A 0 2 0 8:64 A 0 8:32 A 0 E 0 2 0 8:48 A 0 8:16 A 0
Apr 24 14:42:50 | HPMSAP2000G3: disassemble map [2 queue_if_no_path retain_attached_hw_handler 0 2 1 round-robin 0 2 1 8:64 1 8:32 1 round-robin 0 2 1 8:48 1 8:16 1 ]
Apr 24 14:42:50 | HPMSAP2000G3: disassemble status [2 0 0 0 2 1 A 0 2 0 8:64 A 0 8:32 A 0 E 0 2 0 8:48 A 0 8:16 A 0 ]
Apr 24 14:42:50 | sda: (HP:LOGICAL VOLUME) wwid blacklisted
Apr 24 14:42:50 | sda: orphan path, wwid blacklisted
Apr 24 14:42:50 | const prioritizer refcount 1
Apr 24 14:42:50 | tur checker refcount 5
Apr 24 14:42:50 | sdd: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:42:50 | sde: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:42:50 | sdb: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:42:50 | sdc: (HP:p2000 G3 FC) wwid whitelisted
Apr 24 14:42:50 | tur checker refcount 4
Apr 24 14:42:50 | alua prioritizer refcount 4
Apr 24 14:42:50 | tur checker refcount 3
Apr 24 14:42:50 | alua prioritizer refcount 3
Apr 24 14:42:50 | tur checker refcount 2
Apr 24 14:42:50 | alua prioritizer refcount 2
Apr 24 14:42:50 | tur checker refcount 1
Apr 24 14:42:50 | alua prioritizer refcount 1
Apr 24 14:42:50 | unloading alua prioritizer
Apr 24 14:42:50 | unloading const prioritizer
Apr 24 14:42:50 | unloading tur checker
root@PREF33-S-PMOX2:~#

When i try create iscsi new storage i don't know which information / parameters insert in GUI interface.

thx to your help.

Sorry to the longer message
 
you don't have to create a iscsi LUN, you don't have a iscsi SAN, but a fibre channel.
if your multipath is correct, you have a device named HPMSAecc under /dev/mapper/ .....
thuis is the device you have to use for configure LVM......
 
thank's to your response, Menk.

I don't have this, because i dont create lvm.

is necessary use multipath with Fiber Channel ? before installing and configuring multipath, i have the 4 lun in gui interface.


root@PREF33-S-PMOX1:/dev/mapper# ls -lai
total 0
1179 drwxr-xr-x 2 root root 180 Apr 24 15:03 .
1025 drwxr-xr-x 21 root root 4520 Apr 24 15:03 ..
1180 crw------- 1 root root 10, 236 Apr 24 14:07 control
53999 lrwxrwxrwx 1 root root 7 Apr 24 15:03 mpathb -> ../dm-2
20633 lrwxrwxrwx 1 root root 7 Apr 24 14:07 pve-data -> ../dm-5
24834 lrwxrwxrwx 1 root root 7 Apr 24 14:07 pve-data_tdata -> ../dm-4
31915 lrwxrwxrwx 1 root root 7 Apr 24 14:07 pve-data_tmeta -> ../dm-3
112 lrwxrwxrwx 1 root root 7 Apr 24 14:07 pve-root -> ../dm-1
32782 lrwxrwxrwx 1 root root 7 Apr 24 14:07 pve-swap -> ../dm-0

root@PREF33-S-PMOX2:/dev/mapper# ls -lai
total 0
1179 drwxr-xr-x 2 root root 180 Apr 24 15:04 .
1025 drwxr-xr-x 21 root root 4520 Apr 24 15:04 ..
1180 crw------- 1 root root 10, 236 Apr 24 14:08 control
74550 lrwxrwxrwx 1 root root 7 Apr 24 15:04 mpathb -> ../dm-2
13874 lrwxrwxrwx 1 root root 7 Apr 24 14:08 pve-data -> ../dm-5
537 lrwxrwxrwx 1 root root 7 Apr 24 14:08 pve-data_tdata -> ../dm-4
15659 lrwxrwxrwx 1 root root 7 Apr 24 14:08 pve-data_tmeta -> ../dm-3
30748 lrwxrwxrwx 1 root root 7 Apr 24 14:08 pve-root -> ../dm-1
12257 lrwxrwxrwx 1 root root 7 Apr 24 14:08 pve-swap -> ../dm-0
root@PREF33-S-PMOX2:/dev/mapper#

i have 4 entry in /dev/ for the disk (same in interface) /dev/sdb /dev/sdc /dev/sdd /dev/sde

i must create manually (pvcreate...etc...) volume and volume group ?

I dont seeing entry to configure Fiberchannel in the Administrator Guide to proxmox v5.x...it's unreall
 
menk said:
post a lsblk ....
Click to expand...
root@PREF33-S-PMOX1:/dev# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 136.7G 0 disk
├─sda1 8:1 0 1M 0 part
├─sda2 8:2 0 256M 0 part
└─sda3 8:3 0 136.5G 0 part
├─pve-swap 253:0 0 8G 0 lvm [SWAP]
├─pve-root 253:1 0 34G 0 lvm /
├─pve-data_tmeta 253:3 0 80M 0 lvm
│ └─pve-data 253:5 0 78.5G 0 lvm
└─pve-data_tdata 253:4 0 78.5G 0 lvm
└─pve-data 253:5 0 78.5G 0 lvm
sdb 8:16 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
sdc 8:32 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
sdd 8:48 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
sde 8:64 0 7.6T 0 disk
└─mpathb 253:2 0 7.6T 0 mpath
sr0 11:0 1 1024M 0 rom
 
I think you should do following steps:
In the console of PREF33-S-PMOX1 or PREF33-S-PMOX2:
pvcreate /dev/mapper/3600c0ff0001ae601d4d8b05a01000000
vgcreate vmstore1 /dev/mapper/3600c0ff0001ae601d4d8b05a01000000

In the PVE GUI:
Datacenter --> Storage --> Add --> LVM
Id: your friendly name eg. vmstore1
Base storage: Existing volume groups
Volume group: you should see and choose vmstore1
Enable: checkbox - selected by default
Shared: checkbox - you should select it, because your VG will be used by two hosts
Now you will see vmstore1 below local and local-lvm and you can use it as virtual machine disk(s).

According to this document there is no need to use HP Device-Mapper Multipath Enablement Kit (and create /etc/multipath.conf file) beceause of native Linux multipath is now integrated and distributed for most of the latest Linux Releases.
 
Last edited:
Zaqen said:
My 2 servers can see 4 Lun, in reality they are juste one lun, proxmox1 server and proxmo2 server, seeing 4 lun (same number and same capacity) i think because they are 4 fiber channel per server.
Click to expand...

You're seeing four paths because servers are probably connected to the MSA not directly but via SAN switches.
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back