How to install PVE on mdadm RAID1?

Garry

Member
Jan 31, 2022
38
0
6
57
Hi All,

I have 2 NVMe M.2 drive 256 Gb in my server and would like to use them as mirror for OS with PVE.
But as I see PVE installation offers RAID on ZFS only.

What is the best way to install PVE on mdadm RAID1?

Thanks)
 
Thank you for your fast answers!

I need your advice - as I know PVE officially does not support mdadm raid for their installation.
If you want RAID - PVE installer offer to use ZFS (for mirror, for example).


I have read a lot about ZFS (forums and so on) and I've got the opinion that ZFS
1) works fine at enterprise level disks only
2) takes a lot of time to learn and set up

I develop my project only with my own money, and the budget is very limited.
So I have bought a simple consumer NVMe disk, almost the cheapest (looking for a big TBW value).

When I installed PVE on this single NVMe disk it worked well and fast.
To increase reliability, I bought a second similar NVMe consumer disk of the same capacity, but from a different manufacturer (so that the disks do not fail at the same time - I have read this hint at one forum).

It is very tempting to put the system on ZFS, cause it is supported by default and "out of the box".
But I am not sure that my NMVe disks are suitable for ZFS, I afraid that the system will slow.

What do you think is the best way to proceed in my case - create RAID1 by mdadm or use ZFS for it?
 
What SSD models do you got? ZFS got alot of overhead so mdadm might be faster. But ZFS isn't slow with consumer SSDs. More of a problem might be the life expectation of your consumer SSDs but that really depends on your workload. If you got QLC SSDs I wouldn't use them with ZFS. In general I wouldn't use QLC SSDs or SMR HDDs in servers at all, not even in a low budget home server. TLC or MLC consumer SSDs might be fine if you don't got alot of sync writes. Just don't expect them to work forever.
You could just try it and after a month look how much wear SMART is reporting and then extrapolate how long they should survive. If its several years then you are fine. If not you can backup your guests, try mdadm with a fresh PVE install and restore them.
 
Last edited:
What SSD models do you got? ZFS got alot of overhead so mdadm might be faster. But ZFS isn't slow with consumer SSDs. More of a problem might be the life expectation of your consumer SSDs but that really depends on your workload. If you got QLC SSDs I wouldn't use them with ZFS. In general I wouldn't use QLC SSDs or SMR HDDs in servers at all, not even in a low budget home server. TLC or MLC consumer SSDs might be fine if you don't got alot of sync writes. Just don't expect them to work forever.
You could just try it and after a month look how much wear SMART is reporting and then extrapolate how long they should survive. If its several years then you are fine. If not you can backup your guests, try mdadm with a fresh PVE install and restore them.
I have Gigabyte GP-GSM2NE3 256Gb GP-GSM2NE3256GNTD NVMe M.2 disk and a similar one,
It has 300 TBW, 3D TLC technology. The second NVMe M.2 disk is also 3D TLC, 170 TBW.
These two disks I want to use as RAID1 for the system.

For data storage, I bought 3Tb WD Red Pro. As I know it is CMR HDD.

The workload on my project is small now, it is developing and testing period
 
If you do an install like this over a debian install, do all the updates within proxmox work OK? IE do you still need to install standard debian updates on top of the proxmox updates?
 
If you do an install like this over a debian install, do all the updates within proxmox work OK? IE do you still need to install standard debian updates on top of the proxmox updates?
Updates will work fine. You always need to do standard debian updates. Both when installing PVE from the PVE ISO and when installing it on top of Debian.
 
> I need your advice - as I know PVE officially does not support mdadm raid for their installation.

yes, but with the existing infrastructure / installer it shouldn't be too hard to add that as a feature, as mdadm raid for rootfs has the same bells and whistles as zfs raid for rootfs has
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!