Install on single large Raid Lun (~3.8Tb) ?

fortechitsolutions

Renowned Member
Jun 4, 2008
434
46
93
Hi,

I suspect I already know the answer to this, but I wanted to ask just in case:

I've got a server here with a 3Ware 9650se raid controller, and 6 x 1tb sata disks attached. The raid array is configured with a single large raid6 volume with total capacity of ~3.8Tb

It seems this is >2Tb, which appears to be the limit for bootable volume size for most linux installers these days (CentOS, ProxVE included). I gather that GPT partition tables are not supported on boot devices with most linux distros (odd/amazes me?!).

I suspect the only option is to carve up the raid volume into multiple LUNs that are presented to the OS as different 'drives', ie,

sda = 2 tb
sdb = 1.8tb

the problem for me, is that this 3ware card only appears to support 'automatic lun carving' - by which you specify a 'carve size' from 1 to 2000gigs, and then it will chop up the raid into appropriate # of luns, up to a max of 16 in total.

It appears to NOT permit control sizes beyond this trivial slice up measure, ie, my preference would be,

sda = 10gigs for the OS
sdb = everything else for the data slice

however, this raid controller doesn't appear to allow me to do this.


In theory, can I install PVE onto a carved lun arrangement, by which the default setup will use on the SDA / first disk lun presented, and then if I really want to, I can do clever command line fun with LVM to extend the size of the 'data volume slice' (/usr/vz or wherever it is ..) which holds the VM data .. ? Or is this going to be absurdly fiddley and painful .. ?

I really don't like slicing it this way, but due to limitations on the hardware raid controller, I don't seem to have a lot of choice.

Any help/pointers are certainly greatly appreciated.

---Tim Chipman
 
FOOTNOTE: I did a bit more reading, and it seems in the 3ware boot-bios level, in the same area where the raid volume parameters are configured, there is an option to specify a "boot volume" of .. arbitrary specified gigs .. that is carved off the primary raid.

the net result, for instance (I believe)

- total raid6 capacity of ~3.8 Tb
- specify boot volume of ~60gigs (for 'os') will be visible to OS as /dev/sda
- will leave ~(3.8tb less 60gigs) visible to OS as /dev/sdb

If I understand correctly, ProxVE will use both drives in an LVM and put boot on the sda disk ? Or will I still likely have problems with this sort of config .. ?

Any pointers are appreciated.

Thanks!

--Tim
 
Simply create 2 luns (any size < 2TB is OK). Then install pve on sda, and add sdb to the LVM VG /dev/pve/data.

- Dietmar
 
It seems this is >2Tb, which appears to be the limit for bootable volume size for most linux installers these days (CentOS, ProxVE included). I gather that GPT partition tables are not supported on boot devices with most linux distros (odd/amazes me?!).

Does Windows handle that? What about old bios software?
 
Does Windows handle that?
Yes, it can handle GPT partitioned drives since 2003 SP1. And Debian Etch can handle it, except the default bootloader (grub 0.97) will fail to install into MBR.

It is possible to install grub2, but the version in etch-backports has a bug, that prevents it to boot from GPT partitions. Could install from lenny repos, it will upgrade libc6, but it will work. I suggest to use elilo instead.

As for Proxmox it is unable to partition big drive correctly. I belive that debian-installer uses parted for partitioning, which is capable of handling GPT. Also there is a lack of setup console, or i could do everything i needed by hand.
 
Last edited:
Dietmar, thanks for the confirmation / followup (ie, just slice up raid into 2 luns of <2tb each, then add sdb into LVM for the pve data.)

I'll try this today.


Tim
 
It is possible to install grub2, but the version in etch-backports has a bug, that prevents it to boot from GPT partitions. Could install from lenny repos, it will upgrade libc6, but it will work. I suggest to use elilo instead.

grub2 is still very unstable, and there is not even a boot menu. Or is there already a stable release?

elilo is for efi bios? What about old non-efi bios?

As for Proxmox it is unable to partition big drive correctly. I belive that debian-installer uses parted for partitioning, which is capable of handling GPT. Also there is a lack of setup console, or i could do everything i needed by hand.

Switching over to parted/GPT would be no problem, but I am concerned about older bios version?

- Dietmar
 
grub2 is still very unstable, and there is not even a boot menu. Or is there already a stable release?
I think it is stable, because Lenny is already stable and includes it. Can't be sure how much stability it has, i'm not debian team.

I am concerned about older bios version?
I don't know much about efi specifics, but we're using HP hardware with Compaq SmartArray P400 controllers, and booting big arrays works fine.
 
Last edited:
Sorry to get this old thread up again but I just ran into the same problem.
For a backupserver I need a node with 4TB space.

I run eight 1TB disks in raid-10 leaving a partition of 3786GB.
PVE won't install :mad:

Can I still run proxmox and effectivly use ALL space for my container?
If so, how exactly?

Thanks.
 
Hi Erwin,
like Dietmar wrote:
1. Two slices - sda and sdb (sda less 2TB).
2. Normal installation to sda
3. lvm extension:
Code:
pvcreate /dev/sdb
vgdisplay     # look for Free PE
vgextend pve /dev/sdb
vgdisplay    # look already for Free PE
# give /dev/pve/data 1200G more space - or more
lvextend -L +1200G /dev/pve/data        # Attention, 1200G is an example, leave min. 6 GB free, or you get in trouble with your backup
lvdisplay and vgdisplay # to be sure
resize2fs /dev/mapper/pve-data
i think thats all.

Udo
 
Hi Udo,

Thanks a lot, I will try that!
 
Sorry to get this old thread up again but I just ran into the same problem.
For a backupserver I need a node with 4TB space.

I run eight 1TB disks in raid-10 leaving a partition of 3786GB.
PVE won't install :mad:

Can I still run proxmox and effectivly use ALL space for my container?
If so, how exactly?

Thanks.

If you need 1 VM to access 4TB why run PVE? Why not install a native OS?
 
Good morning

As a Debian based system I would expect a "expert" installer, which allows you to do the same installation and partitioning menu as the native Debain system.
Installing first the native OS and then PVE in a second step is just a workaround and not official supported as far as I understood, but i might be wrong.
Regards
tom
 
If you need 1 VM to access 4TB why run PVE? Why not install a native OS?

I want to run more then one container on this server, but a very good reason would be the easy migration to new hardware.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!