Multipath configuration in Proxmox VE 2.x

Petrus4

Member
Feb 18, 2009
249
0
16
Hi Everyone,

I want to create a how-to for Multipath configuration in the wiki, specifically for the Intel Modular Flex Server and dual storage controllers but hopefully the how-to can be generalized to other hardware. I had attempted Multipath under 1.9 with limited success and with 2.0 and Debian Squeeze I hope the multipath support will be better.

If anyone has configured Multipath in PVE 2.X and are willing to share their configs / experiences please post back here. I will also update this post with my experiences.
 
Hi Everyone,

I want to create a how-to for Multipath configuration in the wiki, specifically for the Intel Modular Flex Server and dual storage controllers but hopefully the how-to can be generalized to other hardware. I had attempted Multipath under 1.9 with limited success and with 2.0 and Debian Squeeze I hope the multipath support will be better.

If anyone has configured Multipath in PVE 2.X and are willing to share their configs / experiences please post back here. I will also update this post with my experiences.


I have tried configuring this but it looks like the multipath-tools-boot package in Debian 6 is buggy and not fully developed for grub2. see http://wiki.debian.org/DebianInstaller/MultipathSupport I am going to wait either until they develop it or until I have time to test with grub legacy. In the mean time if anyone has succesfully gotten multipath to work with grub2 and debian squeeze, please let me know.
 
I have tried configuring this but it looks like the multipath-tools-boot package in Debian 6 is buggy and not fully developed for grub2. see http://wiki.debian.org/DebianInstaller/MultipathSupport I am going to wait either until they develop it or until I have time to test with grub legacy. In the mean time if anyone has succesfully gotten multipath to work with grub2 and debian squeeze, please let me know.

News form your test?
It seems that bug it's fixed: Bug Marked as fixed in versions grub2/1.99-12
 
News form your test?
It seems that bug it's fixed: Bug Marked as fixed in versions grub2/1.99-12

the bug in multipath tools is fixed but Grub2 would need to updated to correctly detect multipath partitions so would need to install legacy grub. I have not had a chance to test yet.
 
I've tried to do it on mine and seems to start but I'm not sure if I am on the right way. Tomorrow, if u want, I can post some informations.
 
I've tried to do it on mine and seems to start but I'm not sure if I am on the right way. Tomorrow, if u want, I can post some informations.


Hi Kaya, did have any luck with this and if so, can you share your configuration?
 
Hi , due to high work load I get not enought time to do test. Btw I shared access to a friend and it seems that he had multipath work. Tomorrow I hope to do a simply test by removing the first storage controller and look if compute module still work. Stay tuned :)

(and sorry for english but I'm a little bit tired to write correct. If it isn't clear enought ask me and I will rewrite my post :( )
 
Thanks to a big friend now multipath work with a Debian 7 (wheezy).
Tomorrow, or maybe monday, we will try the same process on proxmox 2.1
 
Small update: the only way that we get multipath work is using a debian 6 standard and install proxmox on.
Now proxmox start without problem, even if SCM1 or SCM2 is removed.
If interested I will post configuration about it.
 
Small update: the only way that we get multipath work is using a debian 6 standard and install proxmox on.
Now proxmox start without problem, even if SCM1 or SCM2 is removed.
If interested I will post configuration about it.

Hi Kaya, I know this thread is old, but if you could post your configuration and steps that would be great! I am also interested in knowing if this can now be done on an existing Proxmox 2.x install, I really don't want to have to rebuild all my servers. Thanks!
 
It's not simple to rimember what I did, even if I wrote it on my internal Wiki :D

Btw, i had to download those packed from testing:

multiarch-support_2.13-33_amd64.deblibmount1_2.20.1-5.1_amd64.deblibfuse2_2.9.0-2_amd64.deblibblkid1_2.20.1-5.1_amd64.debgrub-common_1.99-22.1_amd64.debgrub2-common_1.99-22.1_amd64.debgrub-pc_1.99-22.1_amd64.debgrub-pc-bin_1.99-22.1_amd64.debfuse-utils_2.9.0-2_all.debmount_2.20.1-5.1_amd64.debfuse_2.9.0-2_amd64.deb
there is an order to install due to dependencies, i ran " dpkg -i *.deb " several times

after that i run
apt-get install multipath-tools multipath-tools-boot
with multipath tools installed, you have to edit multipath.conf under /etc/multipath.conf . The following is mine:

blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^(hd|xvd)[a-z][[0-9]*]" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"}devices { device { vendor "Intel" product "Multi-Flex" path_grouping_policy "group_by_prio" getuid_callout "/lib/udev/scsi_id -g -u /dev/%n" prio "alua" path_checker tur path_selector "round-robin 0" hardware_handler "1 alua" failback immediate rr_weight uniform rr_min_io 100 no_path_retry queue features "1 queue_if_no_path" }}multipaths { multipath { wwid 22276000155c0debf alias system }} multipaths { multipath { wwid 222310001555f77ef alias vmdata }}


the wwid is the guid of disk path, u can retrive it with
/lib/udev/scsi_id -g
or with
ls -gG /dev/disk/by-id/

Add the modules into /etc/initramfs-tools/modules by adding the line "scsi_dh_alua" and after that run " update-initramfs -u -k all "

After that I didn't remeber, but on my shell history I see:

37 grub-mkconfig -o /boot/grub/grub.cfg
38 grub-install /dev/dm-0
39 reboot

I'm not sure if it's needed.

Now you have to handle the LVM volume that proxmox create inside fstab, but i don't know how to be helpful.
Please let me know if u need more.


Oh , and user multipath -ll to check if it's running before reboot. AND DON'T be scared if u see a lot of error like
/dev/sdc: read failed after 0 of 4096 at 0: Input/output error
/dev/sdc: read failed after 0 of 4096 at 268435390464: Input/output error
/dev/sdc: read failed after 0 of 4096 at 268435447808: Input/output error

Seems to be normal :D

A.
 
Last edited:
It's not simple to rimember what I did, even if I wrote it on my internal Wiki :D

Btw, i had to download those packed from testing:

multiarch-support_2.13-33_amd64.deblibmount1_2.20.1-5.1_amd64.deblibfuse2_2.9.0-2_amd64.deblibblkid1_2.20.1-5.1_amd64.debgrub-common_1.99-22.1_amd64.debgrub2-common_1.99-22.1_amd64.debgrub-pc_1.99-22.1_amd64.debgrub-pc-bin_1.99-22.1_amd64.debfuse-utils_2.9.0-2_all.debmount_2.20.1-5.1_amd64.debfuse_2.9.0-2_amd64.deb
there is an order to install due to dependencies, i ran " dpkg -i *.deb " several times

after that i run
apt-get install multipath-tools multipath-tools-boot
with multipath tools installed, you have to edit multipath.conf under /etc/multipath.conf . The following is mine:

blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^(hd|xvd)[a-z][[0-9]*]" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"}devices { device { vendor "Intel" product "Multi-Flex" path_grouping_policy "group_by_prio" getuid_callout "/lib/udev/scsi_id -g -u /dev/%n" prio "alua" path_checker tur path_selector "round-robin 0" hardware_handler "1 alua" failback immediate rr_weight uniform rr_min_io 100 no_path_retry queue features "1 queue_if_no_path" }}multipaths { multipath { wwid 22276000155c0debf alias system }} multipaths { multipath { wwid 222310001555f77ef alias vmdata }}


the wwid is the guid of disk path, u can retrive it with
/lib/udev/scsi_id -g
or with
ls -gG /dev/disk/by-id/

Add the modules into /etc/initramfs-tools/modules by adding the line "scsi_dh_alua" and after that run " update-initramfs -u -k all "

After that I didn't remeber, but on my shell history I see:

37 grub-mkconfig -o /boot/grub/grub.cfg
38 grub-install /dev/dm-0
39 reboot

I'm not sure if it's needed.

Now you have to handle the LVM volume that proxmox create inside fstab, but i don't know how to be helpful.
Please let me know if u need more.


Oh , and user multipath -ll to check if it's running before reboot. AND DON'T be scared if u see a lot of error like
/dev/sdc: read failed after 0 of 4096 at 0: Input/output error
/dev/sdc: read failed after 0 of 4096 at 268435390464: Input/output error
/dev/sdc: read failed after 0 of 4096 at 268435447808: Input/output error

Seems to be normal :D

A.

Ok thanks Kaya I will give it a try!
 
SOLVED: Added HOW-TO to WIKI

I created a Multi-Path How-to specific for the INTEL MFSYS25/35 on the PVE Wiki HERE please take a look and provide any feedback for improvements!

Thanks everyone for your help!! Thanks KAYA!
 
Re: SOLVED: Added HOW-TO to WIKI

Great work.
I will take a deep look inside next.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!