Getting started with PVE storage model - software RAID5 array and storage concepts

[aenima99x]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

what is exactly your problem ?

I'm asking how I present a raid array to multiple vm's. What is the proper method? I would like to have (2) mdadm raid5's with lvm be passed through and be available to several different vm's.
Do I setup mdadm/lvm on the proxmox debian install itself? If so, how do I get the vm's to see it?

 

[m.ardito]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

I'm asking how I present a raid array to multiple vm's.

there are many ways, but imho the best way to serve disk space to multiple VMs is use an external shared storage, ie not the same VM host. You could also use the new pve ceph server option, but you would need other 3 pve dedicated pve nodes, minimum. If you can, do not mix hypervisor and disk server functions in pve: use something like a physical nas, with a fast connection to the pve cluster, and use nfs (all usages possible) or iSCSI+LVM (vm disks only) - or both - as protocols; there are other choices, but those are the most usual.

Marco

 

[dominix]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

you setup your RAID/LVM, once the storage is effective you mount it then you create a datastore in proxmox with "images" capabilities.
this is an "alternative" way, or a cheap way, the best way is with dedicated storage like a SAN/NAS/CEPH/NFS.

 

[aenima99x]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

there are many ways, but imho the best way to serve disk space to multiple VMs is use an external shared storage, ie not the same VM host. You could also use the new pve ceph server option, but you would need other 3 pve dedicated pve nodes, minimum. If you can, do not mix hypervisor and disk server functions in pve: use something like a physical nas, with a fast connection to the pve cluster, and use nfs (all usages possible) or iSCSI+LVM (vm disks only) - or both - as protocols; there are other choices, but those are the most usual.

Marco

you setup your RAID/LVM, once the storage is effective you mount it then you create a datastore in proxmox with "images" capabilities.
this is an "alternative" way, or a cheap way, the best way is with dedicated storage like a SAN/NAS/CEPH/NFS.

While I would agree that external storage would be an ideal solution, this is for my home environment and not what I'm looking to do. I've currently got an all-in-one server (ubuntu 14.04) that is acting as my "NAS", HTPC, NFS server, web server, etc. I was hoping to turn the server into a bare-metal hypervisor running all my services in VM's and still keeping the storage function as well.
It sounds like Proxmox might not be the solution for me then, which is too bad because I really like it.

 

[wahmed]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

While I would agree that external storage would be an ideal solution, this is for my home environment and not what I'm looking to do. I've currently got an all-in-one server (ubuntu 14.04) that is acting as my "NAS", HTPC, NFS server, web server, etc. I was hoping to turn the server into a bare-metal hypervisor running all my services in VM's and still keeping the storage function as well.
It sounds like Proxmox might not be the solution for me then, which is too bad because I really like it.

There are simply no reason why you cannot do what you intending to do. Since you mentioned "home environment", i am assuming you do not need node failover, high availability etc etc. I tried to figure out the issue here by reading the earlier postings. If i understand right your facing issue of introducing the RAID to your VMs?

I dont think thats even an issue since Proxmox "can" work with RAID. VMs really dont care whats going on underneath as long as you can store virtual disks on some kind of storage. The simplest thing would be to setup LVM for all your HDDs to increase the local storage size of your proxmox node. Or if you want to keep some kind of RAID, just setup your desired RAID and install Proxmox on top.

 

[aenima99x]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

There are simply no reason why you cannot do what you intending to do. Since you mentioned "home environment", i am assuming you do not need node failover, high availability etc etc. I tried to figure out the issue here by reading the earlier postings. If i understand right your facing issue of introducing the RAID to your VMs?

I dont think thats even an issue since Proxmox "can" work with RAID. VMs really dont care whats going on underneath as long as you can store virtual disks on some kind of storage. The simplest thing would be to setup LVM for all your HDDs to increase the local storage size of your proxmox node. Or if you want to keep some kind of RAID, just setup your desired RAID and install Proxmox on top.

I've made some progress. Just a note that I'm doing all of this on a Proxmox install nested inside KVM running on my laptop before I proceed on my actual server.
I'm trying to closely mimic what I'll eventually be working with and here's what I've done. Please tell me if you see any issues.
I created 8 disks and added them to Proxmox. (Instead of TB size I used GB size.)
Disks 1-4 are 3GB. Created (3) 1GB aligned partitions on each disk
Disks 5-8 are 2GB. Created (2) 1GB aligned partitions on each disk
Ran a pvcreate for each disk partition (/dev/vdb1, /dev/vdb2, etc....)
Ran a vgcreate with all of the pv's (vgcreate Datastore /dev/vdb1, /dev/vdb2, etc.....)

NOW here's where I'm not sure if I'm taking the right direction.....
ran an lvcreate (lvcreate --type raid5 -L 4G -i 2 -n Movies Datastore)
ran mkfs.ext4 /dev/Datastore/Movies
mount -t ext4 /dev/Datastore/Movies /media/Movies
Added new storage in the Proxmox GUI using add:LVM. This allowed me to see the VG and add it.

So it seems that creating the LV was not needed? Would I do this on the individual VM's after I add a disk using the VG?

 

[wahmed]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

There are simple nothing you have to do on individual VMs other than adding more space in the VG and LV in Proxmox node itself. But if you are setting up LVM storage inside VM and want to increase space by adding new virtual disk image without shutting down VM, then yes you will also need to do extend lvm inside VM.

I would not say creating LV was not needed. both VG and LV needs to be created for LVM to work i believe. This rather complicated at the beginning. So if you are confused by now, you are on the right track

Just ask if you need additional clarification.

 

[lpallard]

Re: Getting started with PVE storage model - software RAID5 array and storage concept

There are simple nothing you have to do on individual VMs other than adding more space in the VG and LV in Proxmox node itself. But if you are setting up LVM storage inside VM and want to increase space by adding new virtual disk image without shutting down VM, then yes you will also need to do extend lvm inside VM.

I would not say creating LV was not needed. both VG and LV needs to be created for LVM to work i believe. This rather complicated at the beginning. So if you are confused by now, you are on the right track

Just ask if you need additional clarification.

aenima99x

I am sorry I couldnt reply earlier.. I am constantly trying to find time to post on these forums but life is getting in the way

I have decided to go down the separate storage server route... I just ordered a Supermicro Chassis (16bays) that I will use for storage instead of having VM's and storage on the same physical server...In other words, going in the opposite direction (not putting all my eggs in the same basket).

Have you found your answer yet?

Regarding this topic, I will receive my storage server soon (Supermicro 16bay chassis with dual Xeon L5420 & 48GB ECC RAM) which I believes will provide more horsepower than needed for a nice ZFS storage pool..

I have opted to use ZFS on a 9x HDD setup because of inherent redundency and checksum and bitrot protection... Ideal for my needs! I will also use FreeNAS on this storage server.

I want to use this machine for multiple storage related purposes, these being:

1) Provide a backend storage service for my proxmox server for VM backups
2) Provide storage space for VM files (not the actual VM's but the files generated and used by the VM services, a good example being OpenKM or drupal datastores (thousands of PDF files....)
3) Storage space for software repositories and package repos (local mirrors)
4) Storage for clients (other computers on LAN) to backup their home folders
5) Storage for a cross-LAN NFS share

What should I use to provide reliable and resizeable storage space for these items?

I am thinking of just adding NFS shares with LVM in FreeNAS. This would allow resizing as needed (and as the ZFS pool grows in capacity).

For #1 above, on Proxmox add the NFS share through the WebGUI as a backup volume.

For #2 to #5, add the NFS shares located on the FreeNAS server and locally mount them on the VM's and LAN clients for use..

Is that effective? Better way to do so? In all of that, I am only using NFS technology and nothing else of what FreeNAS or Proxmox are offering (iScsi, Ceph, etc)...

Please share your thoughts!!