Recommendation for Virtual Machine Storage (HW-Raid1 or not)

[pregrcl]

Hello all
Firstly, i'd like to make clear that i am a proxmox beginner and i just started out.
We bought new server hardware to replace our old internal dev-server to run our VMs and i bought a proxmox subscription and installed it. So far so good, i really like proxmox.


My current hardware setup:

• Boot device with 2 960GB NVME SSDs --> During Proxmox installation i used ZFS on these two drives to create a Raid1. It is where Proxmos OS is installed
• MegaRaid 9361-8i, supporting up to 8 SATA SSDs. Currently set up with 2x 4TB SSDs, combined as a Raid 1 Virtual Drive. This is where i want to store the VM's data

Current issues i have noticed

• I noticed ZFS storage displays a warning that Hardware RAIDs should not be used with ZFS. reading on ZFS makes it clear that this is not a good idea.
• When using LVM, i noticed that snapshots are not supported

Question: What is recommended for having a RAID 1 Setup?

• Shall i set the RAID Controller to JBOD (Just a Bunch of Disks) and use a ZFS on it?
• Shall i use the HW Raid Controller to create the Raid1 with the two disks and run a LVM-Thin?
• Any other options?

I am just starting out, so the options confuse me and i would like to have a RAID 1 for redundancy as well as the possibility for snapshots.
What are your recommendations? I am thinking of JBOD-Mode on the raid controller, and to combine the two disks as a Raid 1 ZFS.

Many thanks in advance for any input

Best regards
Claudio

 

[pregrcl]

For the moment i configured it like this:

• HW-Raid-Controller set to JBOD to pass through the disks
• ZFS Raid 1 on the two disks via proxmox
• Storage on this ZFS Raid
• VMs use this storage

Hope this makes sense, at least this way i have snapshots, and Raid is managed by Proxmox.

 

[MagicFab]

Hello,

First of all welcome to the forums Don't forget to put your subcription code in your profile, it helps indicate you have commercial support and staff will respond accordingly.

Thank for the extra details, that setup makes perfect sense : JBOD on the MegaRAID → ZFS mirror in Proxmox → VM disks on that ZFS pool is the recommended configuration.

This gives you redundancy, snapshots, checksumming, scrubs, and avoids the RAID‑on‑RAID issues that happen when ZFS sits on top of hardware RAID.

You’ve configured it the right way.

It's also not dependent on your hardware RAID controller failing ! In fact if you replace or upgrade your system you can transfer your disks and they will work provided you have the same OS config. You can also use the disks in another system for data recovery, for example.

One extra unsollicited advice: test your RAID failing and recovery. Remove a disk, format it, put it back, takes notes, rebuild your system few times - this will make your first disk fail a stress-free event. Oh, and alwats keep a spare disk nearby. Did I mention don't use consumer SSDs ?...

---

Fabián Rodríguez | Le Goût du Libre Inc. | Montreal, Canada | Mastodon
Proxmox Silver Partner, server and desktop enterprise support in French, English and Spanish

 

[alma21]

When using LVM, i noticed that snapshots are not supported

- you can use HW RAID + (thick) LVM/LV with a filesystem (ext4, xfs) and qcow2 virtual disk format for your VMs - this setup supports also snapshots (and thin provisioning)
- or as you already mentioned RAID + lvm-thin .... without add. filesystem layer == faster
both solutions support also non linear snapshots / snapshot trees which afaik is (currently) not supported by ZFS

 

[pregrcl]

Thank you very much for your fast and detailed replies.
I did it like this, because i already prepared it that way and MagicFab confirmed the setup

JBOD on the MegaRAID → ZFS mirror in Proxmox → VM disks on that ZFS pool

I like that all my RAIDs are managed by the proxmox OS (nvme boot disks as well as the vm storage) i think this is easier that having the boot disks managed via ZFS by proxmox and the VM storage via the mainboard. and as magjcFab pointed out, i am independent from the hardware


(I already got spare SSDs for stress reduced emergency replacements)

many thanks
Claudio