Q: Proxmox 2-node cluster with Shared LVM "SAN" (Fibre) Storage array - max vol size?

fortechitsolutions

Renowned Member
Jun 4, 2008
434
46
93
Hi all, quick question (maybe ?)

I am in the process of configuring a storage array for use with a new build proxmox deploy for a client.

- 2 physical nodes running proxmox, latest version, local modest size hardware raid storage for base install of proxmox
- 1 disk array with ~24Tb usable space (Raid6, hotspare, 16x2Tb drives)
- disk array has 2 x fibre ports, so each proxmox node will have a single connection via Fibre to the disk array. LUN(s) on the array will be exposed/visible to both proxVe nodes to facilitate 'shared cluster' access.
- I plan to configure raid volume LUN(s) on the disk array and provision the space to proxmox as shared LVM storage, so live migration is possible easily.

The fun question: I haven't deployed this much shared storage on a proxmox cluster before. (12Tb or less is my current deploy biggest for a project). I'm curious if anyone has strong recommendation about known gotchas in this kind of topology, ie,

- single large LUN of approx 24Tb usable disk space, works well, no gotchas?
- make it visible to both proxVE hosts,
- VM LVMs will be carved off in here by proxmox as disk space is allocated for VMs using this storage pool.

or

- is there a maximum size for the LVM Volume (ie, don't go above 12 Tb per LUN? deploy as 2 x 12Tb instead of 1 x 24Tb ?)
- is there any inherent performance limit which makes LVM get stupid or more PITA to deal with above a certain size ?

I know LVM on 64bit platform supports 'really big' disk capacity in theory. I also know from experience (for example) that ext4 on 64bit is 'meant' to support really big disk capacity but for some platforms, for some support libraries for e2fs utils it means you can't format really big volumes (ie, >12 Tb for example) without having to either compile yourself properly from scratch the disk formatting tools, or other workarounds. So I figure I might as well ask if anyone has strong recommendatoin around this kind of config where I have the choice of deploying a single 24Tb storage volume 'for simplicity' or better to do other things (2x12 Tb, 3 x 8Tb?) for example.

Any comments are greatly appreciated.

Thanks!

Tim Chipman
 
Re: Q: Proxmox 2-node cluster with Shared LVM "SAN" (Fibre) Storage array - max vol s

Hi,
with lvm I don't have such an big VG. If I remember right, there are an thread because of an harmless error message on very big PVs...

Udo
 
Re: Q: Proxmox 2-node cluster with Shared LVM "SAN" (Fibre) Storage array - max vol s

Hi Udo, thanks for the quick reply! Probably it is safe to deploy with 2 x 12Tb and I will not get any 'harmless error messages' ?

I am fairly positive the client does not need to allocate big (ie 12Tb or larger) disk to any single VM so really there is minimal benefit to making a single gigantic LVM instead of 2 x smaller ones -- I think.

Thanks!

Tim
 
Re: Q: Proxmox 2-node cluster with Shared LVM "SAN" (Fibre) Storage array - max vol s

Hi Tim,
don't find the thread... perhaps you should simply try the 24TB-LVM and see what happens.

If you also see error messages, you can go for 2 volumes.

Udo
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!