Problem with ProxMox/Linux/Openfiler -and- Adaptec 5805z Controller Card

[oeginc]

I'm hoping someone here might have the answer to this problem, it's been driving me nuts for a week now.

I have an Adaptec 5805z controller card installed in a machine, the machine has

1 x 120GB 7,200 RPM SATA2
7 x 500GB 7,200 RPM SATA2

I have drive #0 setup as a "Volume" for my boot drive.
I have drive #1-6 setup as a RAID-10 (stripe of 3 sets of mirrored drives).
I have drive #7 setup as a global hotspare.

Now, if I go into the RAID manager and ONLY create the BOOT drive, reboot, install, reboot off the hard drive, everything works fine.

If I configure my RAID-10 DATA volumes and reboot, everything still works fine.

If I initialize my RAID-10 DATA volumes in any way (thru linux), Ie: start using them for iSCSI data, or partition & format it, etc. I can't boot anymore. I get to the "Verifying DMI Pool Data... Boot from CD: None" and then it just hangs. It never even gets to grub...

If I go back into the RAID manager and re-init all the drives and recreate the RAID-10 I can boot again.

I'm banging my head here... I've already updated the motherboard BIOS to the latest, and the Adaptec BIOS to the latest... It didn't fix the problem.

 

[JustaGuy]

"Verifying DMI Pool Data... Boot from CD: None"

So I assume this is an OF PVE talks to, and you're reaching that same failure point with or without the CD providing the kernel, and it's OF that isn't booting, not PVE, correct?

I'm guessing kernel issues.
Something along the lines of bad superblocks- it's the data populating the blocks, not the availability of blocks.

Are you able to install various kernels to test with prior to beginning to use the RAID volume when it fails?
Do these kernels fail at the same point?

How about another OS, how does it do communicating with the controller in the same situation?

Can the array be made use of in another capacity? To rule out the controller?
What if the data volumes have something installed to it and it's booting off those?
Just for troubleshooting's sake.


I don't see how it would matter, but for lack of any other option I'd partition them <2TB just to see.
And also I'm no expert, just brainstorming on a forum hoping something helps you move forward.

 

[oeginc]

So I assume this is an OF PVE talks to, and you're reaching that same failure point with or without the CD providing the kernel, and it's OF that isn't booting, not PVE, correct?

I'm guessing kernel issues.
Something along the lines of bad superblocks- it's the data populating the blocks, not the availability of blocks.

Are you able to install various kernels to test with prior to beginning to use the RAID volume when it fails?
Do these kernels fail at the same point?

How about another OS, how does it do communicating with the controller in the same situation?

Can the array be made use of in another capacity? To rule out the controller?
What if the data volumes have something installed to it and it's booting off those?
Just for troubleshooting's sake.


I don't see how it would matter, but for lack of any other option I'd partition them <2TB just to see.
And also I'm no expert, just brainstorming on a forum hoping something helps you move forward.

I'm talking about the physical hardware node, and that is what has the RAID in it. It doesn't matter
what operating system I install on it, if the data raid is partitioned - it won't boot. If I go into the RAID
manager and re-init, re-create the partition I can boot fine again. As soon as I use the data partition for
something it won't boot anymore until I go back into the manager and re-init, re-create the RAID.

P.S. The data partition is only 1.36TB (1.5TB theoretical) because it's 6 500GB drives in RAID-10.

 

[udo]

I'm talking about the physical hardware node, and that is what has the RAID in it. It doesn't matter
what operating system I install on it, if the data raid is partitioned - it won't boot. If I go into the RAID
manager and re-init, re-create the partition I can boot fine again. As soon as I use the data partition for
something it won't boot anymore until I go back into the manager and re-init, re-create the RAID.

P.S. The data partition is only 1.36TB (1.5TB theoretical) because it's 6 500GB drives in RAID-10.

Hi,
look for the diskorder of the bios. I think that grub find sda1 (/boot) without raid, but the raid is sda and your boot-device sdb?!
If you change the raid in the bios to "sdb" it should work. If you have an empty sata connector on the raid, you can put your bootdisk as "pass through-device" on the raidcontroller and change the diskorder inside the controller-bios (but i'm no expert with adaptec-controller).

Udo

 

[oeginc]

Hi,
look for the diskorder of the bios. I think that grub find sda1 (/boot) without raid, but the raid is sda and your boot-device sdb?!
If you change the raid in the bios to "sdb" it should work. If you have an empty sata connector on the raid, you can put your bootdisk as "pass through-device" on the raidcontroller and change the diskorder inside the controller-bios (but i'm no expert with adaptec-controller).

Udo

All of the drives are on the raid card.
The first drive (drive #0) on the raid card is setup as a pass-thru drive formatted & installed with the OS.
The bios is set to boot off of the raid card.
The raid card is set to boot off the first drive.

Everything works fine if I don't have a partition on the "data" raid volume.
As soon as I partition the data raid, it won't boot anymore.
If I delete the partition information - it boots again.

Very strange!

 

[udo]

All of the drives are on the raid card.
The first drive (drive #0) on the raid card is setup as a pass-thru drive formatted & installed with the OS.
The bios is set to boot off of the raid card.
The raid card is set to boot off the first drive.

Everything works fine if I don't have a partition on the "data" raid volume.
As soon as I partition the data raid, it won't boot anymore.
If I delete the partition information - it boots again.

Very strange!

Hi,
if you create a partition on data and boot then a live cd (like grml). Which disk/partition is sda?
Please post the output of "fdisk -l".

What mean exactly don't boot? Do you see the grub selection or only a "_"?

Udo