[SOLVED] best way to store proxmox?

Dunuin

Distinguished Member
Jun 30, 2020
14,793
4,630
258
Germany
Hi,

I would like to encrypt my data pools and have 4 options to store proxmox. What would you prefer? What will be the most stable one on the long term if you need to upgrade the distribution or to swap a failed drive?

1.) just install ProxmoxVE as raidz1 on the first partitions of a 5x 200GB SSD (Intel DC S3700/S3710) zraid1 pool and only use 5%/40GB or something like that for Proxmox and then manally create partitions on the 95% free space of every SSD and create a encrypted 760GB zraid1 pool for VMs. There is no room for swap so I would need to create a luks encrypted swap partition on a single 4 TB HDD I wanted to use to store the RAM dumps for snapshotting.

2.) install ProxmoxVE as a zfs mirror on two 100GB SSDs (Intel DC S3710) and leaving 64GB free so I could later use mdraid and luks to create a mirrored and encrypted 64GB swap partition on that pool. I would have the complete 5x 200GB SSD (Intel DC S3700/S3710) zraid1 pool only for VM images and the complete 4TB HDD for RAM dumps from snapshots.

3.) activate onboard raid1 for the two 100GB SSDs and install ProxmoxVE as xfs without data and swap LVs on the first 36 GB of the raidset. Then I could later create a encypted 64GB swap partition on the free space of that raidset. Again a complete zraid1 5x 200GB SSD pool for VM images and a complete 4TB HDD for RAM dumps.

4.) install Debian 10.5 using mdraid to mirror the two 100GB SSDs. Ontop of that mdraid a LVM with a small unencrypted ext4 LV for booting and a big encrypted VG for the root xfs LV and swap LV. This way I would get a complete system encryption and I could use dropbear to unlock the server via SSH. But then I would need to install Proxmox ontop of that debian an I worry that this will cause problems on the long term always manipulation the kernel. Again I would have a complete 5x 200GB zraid1 pool for VMs and a 4TB HDD for RAM dumps.

Running linux from a ZFS dataset is fairly new but great on powerloss because it doesn't need backup battery because of the atomic writes. But do you think it is more stable to use mdraid or onboard raid? What of the 3 raid types would you prefer?
 
Hi,

version 1-2 will kill your ZFS performance.
ZFS should have control over the complete disk if you are interested in a good performance.
Swap on the same disk will be the worst scenario for this case.

If you have an HW controller and no Fakeraid version 3 is the recommended and also supported.

Proxmox VE does not support MD-raid.
But sure you can use it.
But keep in mind is not tested and problems related to MD-raid will not be supported by Proxmox.

Also consider ZFS can encrypt datasets, not pools.
This means you can install Proxmox VE on ZFS and encrypted the VM data without creating a new pool.
 
I chose option 4 and it worked without problems so far.

I installed Debian 10.5 to 2x 100GB Intel S3700 SSDs:
sda: 1MB for grub reserved
sdb: 1 MB for grub reserved
sda1 and sdb1: 512 MB -> mdraid -> ext4 as /boot mountpoint
sda2 and sdb2: 99.5 GB -> mdraid -> LUKS -> LVM -> VG -> 22GB LV via xfs as root mountpoint + 64GB LV as swap + 11 GB left free for LVM snapshots
I copied the bootloader after installation from sda to sdb, installed dropbear-initramfs to be able to unlock the LUKS full system encryption via SSH and installed proxmoxve.

5x 200GB Intel S3710 SSD as raidz1 zfs pool with encrypted dataset for all the VMs.
1x 4TB HDD zfs pool with encrypted dataset for RAM dumps to snapshot VMs.

Great that it is so easy to install proxmox on a custom debian installation! :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!