Installing Proxmox on a server with physical RAID

[voidindigo]

I'm rebuilding my Proxmox server from v5.X to the latest 8.X. My system, however, has multipole hardware raid-ed disks. ZFS says it doesn't support being installed on physical RAID and I need to enable snapshots in the VMs once the system is complete. So, I want to use BTRFS as the base FS.

What I want as the OS disk is actually 2 physical ~930GB drives in a RAID1 configuration
What I want as the image disk is actually 4 physical ~5.54TB drives in a RAID5 configuration

When I install Proxmox is doesn't have an option (that I saw) that allows me to specify manual partitioning. As a test I chose RAID0 and it defaulted to 930GB of space (which I hoped meant it was choosing only the first RAID volume) ... and that caused it to take the first RAID volume AND a ~930GB slice from the second RAID volume. So now I have software RAID0 spanning across 2 physical RAID volumes with very different configurations.

Is there a way to manually select where and how to partition the system during install? Am I missing something?

Thanks!

 

[Dunuin]

ZFS says it doesn't support being installed on physical RAID and I need to enable snapshots in the VMs once the system is complete. So, I want to use BTRFS as the base FS.

Not sure about btrfs but software raid (even if it is just a single disk raid0) on top of HW raid usually isn't a good option. Why not just a simple LVM-Thin or ext4/xfs with qcow2? Both would allow snapshotting too and probably perform better and not in "feature preview" stage.

When I install ProxMox is doesn't have an option (that I saw) that allows me to specify manual partitioning.

For that you could install a Debian 12 (and the Debian installer allows you to partition anything you want...except for ZFS...) and you can then turn that into a PVE8 later: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm

As a test I chose RAID0 and it defaulted to 930GB of space (which I hoped meant it was choosing only the first RAID volume) ... and that caused it to take the first RAID volume AND a ~930GB slice from the second RAID volume. So now I have software RAID0 spanning across 2 physical RAID volumes with very different configurations.

Is there a way to manually select where and how to partition the system during install? Am I missing something?

Only set your system disks in the PVE installer. You can later partition your VM/LXC storage via the webUI.

 

[Azunai333]

Not sure about btrfs

btrfs RAID 1 or 0 can be chosen with the installer if you select "Options" when he asks for the Target Harddisk.

 

[voidindigo]

Not sure about btrfs but software raid (even if it is just a single disk raid0) on top of HW raid usually isn't a good option. Why not just a simple LVM-Thin or ext4/xfs with qcow2? Both would allow snapshotting too and probably perform better and not in "feature preview" stage.

This was my thought as well, raid on top of raid is a bad idea... but the btrfs install didn't seem to allow for non-raid. I had thought about LVM-thin ... I might go with that, given the options. I wasn't aware I could do snapshotting on the ext4/xfs filesystem... I thought it required qcow2 + a filesystem that supports snapshots as well... that's my bad, thanks for that!

For that you could install a Debian 12 (and the Debian installer allows you to partition anything you want...except for ZFS...) and you can then turn that into a PVE8 later: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm

From what I read online, this can be very problematic... is that not the case?

Only set your system disks in the PVE installer. You can later partition your VM/LXC storage via the webUI.

Ok this makes sense. I did manage to get the remainder of the volume configured by partitioning from the command line, then using the UI to configure it as storage. But again... raid on top of raid = badness.

Ok, perhaps it makes sense to drop back and re-do this as XFS or LVM-thin on the physical RAID (since I am kind of stuck with that) & use qcow2 for snapshots... it sounds like that makes the most sense given my configuration, yes?

 

[Dunuin]

I wasn't aware I could do snapshotting on the ext4/xfs filesystem... I thought it required qcow2 + a filesystem that supports snapshots as well... that's my bad, thanks for that!

When using a "Directory" storage it can point to any folder on any mounted filesystem. The snapshotting is then part of qcow2 format image files and will work no matter what the underlaying storage may be. When using ext4/xfs and "raw" format image files there won't be snapshotting.
When using LVM-Thin your virtual disks will be created as block devices (thin volumes) and the LVM native snapshotting feature will be used.

 

[voidindigo]

Gotcha, thank you!

 

[_gabriel]

Lvmthin is the default storage for VM when choose ext4 or xfs.
ext4/xfs will be only for pve os itself (about 6 GB used) and iso storage / templates

 

[voidindigo]

Lvmthin is the default storage for VM when choose ext4 or xfs.
ext4/xfs will be only for pve os itself (about 6 GB used) and iso storage / templates

This is what I remember from PVE 5 as well, but I could not do snapshots on that setup. I will have to be more careful about how I set up the storage this time around... it sounds like I did something odd during the installation.