Install on single large Raid Lun (~3.8Tb) ?

[fortechitsolutions]

Hi,

I suspect I already know the answer to this, but I wanted to ask just in case:

I've got a server here with a 3Ware 9650se raid controller, and 6 x 1tb sata disks attached. The raid array is configured with a single large raid6 volume with total capacity of ~3.8Tb

It seems this is >2Tb, which appears to be the limit for bootable volume size for most linux installers these days (CentOS, ProxVE included). I gather that GPT partition tables are not supported on boot devices with most linux distros (odd/amazes me?!).

I suspect the only option is to carve up the raid volume into multiple LUNs that are presented to the OS as different 'drives', ie,

sda = 2 tb
sdb = 1.8tb

the problem for me, is that this 3ware card only appears to support 'automatic lun carving' - by which you specify a 'carve size' from 1 to 2000gigs, and then it will chop up the raid into appropriate # of luns, up to a max of 16 in total.

It appears to NOT permit control sizes beyond this trivial slice up measure, ie, my preference would be,

sda = 10gigs for the OS
sdb = everything else for the data slice

however, this raid controller doesn't appear to allow me to do this.


In theory, can I install PVE onto a carved lun arrangement, by which the default setup will use on the SDA / first disk lun presented, and then if I really want to, I can do clever command line fun with LVM to extend the size of the 'data volume slice' (/usr/vz or wherever it is ..) which holds the VM data .. ? Or is this going to be absurdly fiddley and painful .. ?

I really don't like slicing it this way, but due to limitations on the hardware raid controller, I don't seem to have a lot of choice.

Any help/pointers are certainly greatly appreciated.

---Tim Chipman

 

[fortechitsolutions]

FOOTNOTE: I did a bit more reading, and it seems in the 3ware boot-bios level, in the same area where the raid volume parameters are configured, there is an option to specify a "boot volume" of .. arbitrary specified gigs .. that is carved off the primary raid.

the net result, for instance (I believe)

- total raid6 capacity of ~3.8 Tb
- specify boot volume of ~60gigs (for 'os') will be visible to OS as /dev/sda
- will leave ~(3.8tb less 60gigs) visible to OS as /dev/sdb

If I understand correctly, ProxVE will use both drives in an LVM and put boot on the sda disk ? Or will I still likely have problems with this sort of config .. ?

Any pointers are appreciated.

Thanks!

--Tim

 

[dietmar]

Simply create 2 luns (any size < 2TB is OK). Then install pve on sda, and add sdb to the LVM VG /dev/pve/data.

- Dietmar

 

[dietmar]

It seems this is >2Tb, which appears to be the limit for bootable volume size for most linux installers these days (CentOS, ProxVE included). I gather that GPT partition tables are not supported on boot devices with most linux distros (odd/amazes me?!).

Does Windows handle that? What about old bios software?

 

[gm2x]

Does Windows handle that?

Yes, it can handle GPT partitioned drives since 2003 SP1. And Debian Etch can handle it, except the default bootloader (grub 0.97) will fail to install into MBR.

It is possible to install grub2, but the version in etch-backports has a bug, that prevents it to boot from GPT partitions. Could install from lenny repos, it will upgrade libc6, but it will work. I suggest to use elilo instead.

As for Proxmox it is unable to partition big drive correctly. I belive that debian-installer uses parted for partitioning, which is capable of handling GPT. Also there is a lack of setup console, or i could do everything i needed by hand.

 

[fortechitsolutions]

Dietmar, thanks for the confirmation / followup (ie, just slice up raid into 2 luns of <2tb each, then add sdb into LVM for the pve data.)

I'll try this today.


Tim

 

[dietmar]

It is possible to install grub2, but the version in etch-backports has a bug, that prevents it to boot from GPT partitions. Could install from lenny repos, it will upgrade libc6, but it will work. I suggest to use elilo instead.

grub2 is still very unstable, and there is not even a boot menu. Or is there already a stable release?

elilo is for efi bios? What about old non-efi bios?

As for Proxmox it is unable to partition big drive correctly. I belive that debian-installer uses parted for partitioning, which is capable of handling GPT. Also there is a lack of setup console, or i could do everything i needed by hand.

Switching over to parted/GPT would be no problem, but I am concerned about older bios version?

- Dietmar

 

[gm2x]

grub2 is still very unstable, and there is not even a boot menu. Or is there already a stable release?

I think it is stable, because Lenny is already stable and includes it. Can't be sure how much stability it has, i'm not debian team.

I am concerned about older bios version?

I don't know much about efi specifics, but we're using HP hardware with Compaq SmartArray P400 controllers, and booting big arrays works fine.

 

[Erwin123]

Sorry to get this old thread up again but I just ran into the same problem.
For a backupserver I need a node with 4TB space.

I run eight 1TB disks in raid-10 leaving a partition of 3786GB.
PVE won't install

Can I still run proxmox and effectivly use ALL space for my container?
If so, how exactly?

Thanks.

 

[udo]

Hi Erwin,
like Dietmar wrote:
1. Two slices - sda and sdb (sda less 2TB).
2. Normal installation to sda
3. lvm extension:

pvcreate /dev/sdb
vgdisplay     # look for Free PE
vgextend pve /dev/sdb
vgdisplay    # look already for Free PE
# give /dev/pve/data 1200G more space - or more
lvextend -L +1200G /dev/pve/data        # Attention, 1200G is an example, leave min. 6 GB free, or you get in trouble with your backup
lvdisplay and vgdisplay # to be sure
resize2fs /dev/mapper/pve-data

i think thats all.

Udo

 

[Erwin123]

Hi Udo,

Thanks a lot, I will try that!

 

[rugby]

Sorry to get this old thread up again but I just ran into the same problem.
For a backupserver I need a node with 4TB space.

I run eight 1TB disks in raid-10 leaving a partition of 3786GB.
PVE won't install

Can I still run proxmox and effectivly use ALL space for my container?
If so, how exactly?

Thanks.

If you need 1 VM to access 4TB why run PVE? Why not install a native OS?

 

[sumsum]

Good morning

As a Debian based system I would expect a "expert" installer, which allows you to do the same installation and partitioning menu as the native Debain system.
Installing first the native OS and then PVE in a second step is just a workaround and not official supported as far as I understood, but i might be wrong.
Regards
tom

 

[Erwin123]

If you need 1 VM to access 4TB why run PVE? Why not install a native OS?

I want to run more then one container on this server, but a very good reason would be the easy migration to new hardware.

 

[JustaGuy]

...