ProxMox on a direct attached shared storage system

M

mjoconr

Renowned Member
Dec 5, 2009
88
1
73
Hi All

My reading of the documentation would seem to indicate that ProxMox is not suited at this time to using on a system which uses direct attached shared storage. (eg Intel Modular Server)

These systems have a miniSAS connection between the CPU board (blades or individual servers) and a storage controller. The storage controller then has a direct SAS connection to the hard drives.

Am I correct ?

If not could someone please explain how best to set it up to support KVM guests.

Thanks
Mike
 
Ok, after even more reading I decided to try something.

I installed proxmox on one of the compu blades, then created a 200G raid 5 virtual drive which I added as a shared device

proxmox-123:/var/lib/vz/template/iso# lsscsi
[0:0:0:0] disk Intel Multi-Flex 0304 /dev/sda
[0:0:0:11] disk Intel Multi-Flex 0304 /dev/sdb
[1:0:0:0] cd/dvd Virtual CDROM 1.00 /dev/sr0
[1:0:0:1] disk Virtual Floppy 1.00 /dev/sdc
proxmox-123:/var/lib/vz/template/iso#

Device [0:0:0:11] is the shared storage.

I then created a lvm using the instructions in the wiki for local drives.

proxmox-123:/var/lib/vz/template/iso# vgdisplay
--- Volume group ---
VG Name vgbulk2
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 200.00 GB
PE Size 4.00 MB
Total PE 51199
Alloc PE / Size 8192 / 32.00 GB
Free PE / Size 43007 / 168.00 GB
VG UUID vef70z-gSZt-PZMF-3Sg2-LmLF-QoHi-vPTU8A

I then added this as a LVM group in proxmox and ticked the shared option.

I build a ubuntu kvm guest.

I then build a new compu node with proxmox making sure it had access to the shared scsi device.

Again I added the LVM group in proxmox.

I then ran the cluster command, and was able to migrate a running guest.

MY Questions:
1. Was this the right way ?
2. Am I safe, does proxmox protect the storage in the LVM ?
3. Which web interface should I use for managment as the second is giving errors now.

Thanks
Mike
 
Hi Mike,

How is you testing of the flex server going? I am about to start testing on the one we have over here.

I see you configured iSCSI for storage. Are you implementing any NIC bonding? If so can you share your configuration? and any results?

Also any other new findings / issues you may have run into while testing?

thanks,

Petrus
 
Petrus4 said:
How is you testing of the flex server going? I am about to start testing on the one we have over here.
Click to expand...
Its going very well, no issues at all

Petrus4 said:
I see you configured iSCSI for storage. Are you implementing any NIC bonding? If so can you share your configuration? and any results?
Click to expand...
I have no iSCSI, its all connected via MiniSAS, via a storage controller.

Petrus4 said:
Also any other new findings / issues you may have run into while testing?
Click to expand...

I've had no issues at all with the Intel Modular Server running ProxMox

Cheers
Mike
 
Hey there,

Can you share with me how you installed Proxmox-Ve on the intel flex. I am not geting 1.5 iso image to install keep getting I/0 error sda

Were there any changes that you had to make during your installation?
 
Hi Petrus
Petrus4 said:
Can you share with me how you installed Proxmox-Ve on the intel flex. I am not geting 1.5 iso image to install keep getting I/0 error sda
Click to expand...

I did nothing special, I installed with the 1.4 iso. I downloaded the ISO then attached it to the JAVA console APP via dedirect IO. Then configure a 10Gig raid 5 image with the boot option set and gave it LUN 0, from there it just worked.

Petrus4 said:
Were there any changes that you had to make during your installation?
Click to expand...
[/QUOTE]
Nothing that seemed out side of the normal settings.

Mike
 
mjoconr said:
I think you can select the kernel at installed time ?
If not give 1.4 a try

Mike
Click to expand...

Hey Mike,

installing with the 1.4 iso worked great! Thanks for the tip. I upgraded from there and now have 1.5 with the 2.6.24-8-pve kernel.

I started a thread on any issues I run into while installed on the MFSYS25 system here.

The folks who sold the flex server to us recommended two storage controllers as this is what Intel recommends for optimal redundancy in case of storage controller failure.
As I understand it, if the storage controller fails it may take days to rebuild the new storage controller logs etc if the storage controller needs to be replaced. This down time is of course unacceptable in a production virtualized environment with many server running on the flex server. With two storage controller the fail-over is in seconds. I don't believe you had two storage controllers in your specs... but do you know if this dual storage controller fail-over is supported in ProxMox-VE ? I believe I read somewhere that the OS needed to be compatible with dual storage controller fail-over.
 
Last edited:
Petrus4 said:
With two storage controller the fail-over is in seconds. I don't believe you had two storage controllers in your specs... but do you know if this dual storage controller fail-over is supported in ProxMox-VE ? I believe I read somewhere that the OS needed to be compatible with dual storage controller fail-over.
Click to expand...

Its called multipath and can be made to work with proxmox but the install is manual and will need you to identify the correct dev entries and also disable the normal dev entries.

Find a good HowTo and give it a try.

This one maybe http://anothersysadmin.wordpress.co...n-and-scsi-multipathing-with-multipath-tools/

Mike
 
Last edited:
Thanks for the How-To it looks like this is for iSCSI, so are you saying multipath would also work with the two SAS storage controllers in the intel flex server?
 
I've never done it but after reading the install doc for Enterprise Redhat on this hardware I'm sure that multipath package is the correct tool
 
mjoconr said:
I've never done it but after reading the install doc for Enterprise Redhat on this hardware I'm sure that multipath package is the correct tool
Click to expand...

Yes after reading a bit also It looks like there is a special driver for Suse Enterprise and Redhat that will allow failover to the second storage controller. I will see if I can get this to work on Debian. It looks like they are using multipath and device mapper packages but also that there is a special driver: LSI MPT SAS driver only for SuSE
Hope there is something for Debian also.

What are your thoughts on having two storage controllers on the flex system?
 
The special driver is only mentioned for a Suse system so I think its just a case of the Suse system having an old kernel.

Having two controllers would be nice, but at $2500 or so Aussie it was little out side the budget this time.

If you can get it working then it will be on the list for the next upgrade (two more compute modules and doubling the ram for all the other compute modules)

Cheers
Mike
 
Last edited:
Hi Petrus/Mike,

I'm also planning to run Proxmox on a dual-controller intel modular server (no hardware available yet). Have you already got the controller failover working?

Thanks!

David
 
Hi David,

I have not had the time yet to allocate to configuring this. Will post here when I do. In the mean-time if you do get it to work please post back here. :)

Petrus
 
Petrus4 said:
Will post here when I do. In the mean-time if you do get it to work please post back here. :)
Click to expand...

I sure will, but I'm waiting for the hardware. I hope next week I will be able to try it.

In the meantime, if you have suggestions as to what to do/try (i.e. what you haven't had time for but maybe intended to), they're also welcome :)

I was thinking to start and see what the built-in kernel modules already support, and if necessary try Debian Lenny's multipath-tools package (but I don't expect the configuration to be easy).

Best regards,
David
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back