Adding Raid10 LVM storage shows 100% full on GUI.

[Deli Veli]

Goal:
Create 4 drive RAID10 over LVM and add that storage to Proxmox.

Problem:
When I add raid10 lvm storage, Proxmox gui shows %100 full on the storage && shows ~4T of space instead of respecting raid10 mirrors and show ~2T of space. Creating a mount point and mounting to a folder on root works.

This does not work - it shows 100% full:
Steps taken below;

pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
vgcreate my_vol_grp /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
lvcreate --type raid10 -m 1 -i 2 -l 100%FREE -n lvm_raid10 my_vol_grp

Now add that LVM as storage on Proxmox Gui;


This works:
Steps taken below;

pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
vgcreate my_vol_grp /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
lvcreate --type raid10 -m 1 -i 2 -l 100%FREE -n lvm_raid10 my_vol_grp
mkfs.ext4 /dev/my_vol_grp/lvm_raid10
mkdir /mnt/lvm_raid10_mount/
mount /dev/my_vol_grp/lvm_raid10 /mnt/lvm_raid10_mount/

Now add that folder as storage on Proxmox Gui;


I would like to use LVM storage added directly instead of using a folder and mount point. Also please notice that ~4T is being shown on LVM storage added. It should be ~2T after Raid10. What could be the issue?

 

[dcsapak]

if you want to use lvm as a storage, you should not create a lv but let proxmox handle the lv creation (so each vm disk gets its own lv)

 

[Deli Veli]

Hi @dcsapak thank you for getting back.

I have 4x 1T NVMe ssd. I have to create Raid10 for redundancy. Another goal is to get the best performance from those PCI ssds. How else can I accomplish this without above steps?

 

[Deli Veli]

Any clue on how to accomplish this? Creating Raid 10 with LVM and have Proxmox not to recognize this as 100% full?

 

[mailinglists]

Just an Idea. While slower, you can have ZVOLs on all kinds of ZFS RAID. Maybe try ZFS?

 

[Deli Veli]

Yes gave a shot to ZFS. Performance decreased drastically. I know that I am missing something very trivial with the Proxmox showing %100 allocated when it is not. Just I am not sure if it is a Proxmox bug or my implementation below is not correct.

pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
vgcreate my_vol_grp /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
lvcreate --type raid10 -m 1 -i 2 -l 100%FREE -n lvm_raid10 my_vol_grp

Then added LVM from the GUI but it shows 100% allocated on the GUI

 

[czechsys]

Your problem is, you don't understand what LV vs VG is.
In case 1]:
- you created LV as raid 10, mounted to pve, but without FS its not usable for any filedata
in case 2]
- you created LV as raid 10 with FS, mounted as directory to pve, thats usable for any filedata

You want raid10 with lvm? I don't know lvm raids, but usually i do:
a] HW raid 10 -> vg -> mounted to pve as lvm or
a+] created lv -> format -> mounted as directory to pve
b] mdraid 10 -> vg -> mounted to pve as lvm or
b+] created lv -> format -> mounted as directory to pve

 

[Deli Veli]

Unfortunately hardware raid is not an option since we use PCI ssds. It has got to be a software raid. High Performance is a must. Gave a shot to zraid but it was extremely slow. LVM raid seems a lot faster. However, ideally what I would like to do is to create the LVM raid and then add this to pve. Is it possible?

 

[mailinglists]

I see no reason why it would not work. One can create and add all kinds of (different) storage using Datacenter -> Storage gui. There are options for LVM and Thin LVM. Just create it correctly and add it. No need to mount it manually unless you want to add it as directory storage.

 

[Deli Veli]

I see no reason why it would not work. One can create and add all kinds of (different) storage using Datacenter -> Storage gui. There are options for LVM and Thin LVM. Just create it correctly and add it. No need to mount it manually unless you want to add it as directory storage.

Simple LVM addition through the gui is simple and I had done it in the past. It works just fine. However, adding lvm via gui does not give you the option of creating the software raid. As mentioned, software raid is a must.

Lets try to find the answer a different way. The goal is to create LVM software raid 10 and add that storage to Pmox. Now below is what I tried to do and it does not work. After issuing commands below, I go to Pmox gui and try to add an LVM and it shows 100% full disk space with ~4T allocated. When it should show ~1.8T because of the software raid 10.

pvcreate /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
vgcreate my_vol_grp /dev/nvme0n1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1
lvcreate --type raid10 -m 1 -i 2 -l 100%FREE -n lvm_raid10 my_vol_grp

I also tried to create an ext4 FS based on that LVM and try to add it on Pmox gui and I get the same result. ~4T shown and 100% full. You can see the image on the first post to check what I see on gui.

What is the correct implementation of this task. How can I create LVM Software Raid 10 using 4 drives, and add that as LVM to proxmox to be used?