Proxmox VE installation and 3 disks

G

gusto

Well-Known Member
Feb 10, 2018
101
3
58
25
I want to buy a mini PC (ALLIWAWA, Ninkear, GMKtec, Blitzwolf, AOOSTAR ...). Some mini PCs have 3x NVMe slots.
I will install proxmox on an NVMe disk (EXT4).
The other 2 NVMe disks will serve as data storage for LXC/KVM, e.g. nextcloud data, plex data, etc ...
These 2 disks will be a ZFS Raid1 mirror.
In case of reinstalling Proxmox VE, the data on the disks with ZFS will not be corrupted.
Is this solution suitable?

I was also looking for a mini PC with 4x NVMe, but they are few and very expensive.


I even considered just 2 disks, but when reinstalling Proxmox VE I would always lose data.
 
I don't have data on proxmox VE. If something happens to the NVMe disk where PVE will be, I will replace it and install a new one.
If the disk where the data will be fails, I will lose all the data. If there are 2 disks in ZFS, then there is a small probability that both disks will fail together.
Ideally, there would be 4 NVMe slots, but even mini PCs are expensive.
I only use PVE as a homelab, which means that if the disk with Proxmox fails, I can quickly replace it.
 
gusto said:
Is this solution suitable?
Click to expand...
Sure.

I am not sure if I would recommend the following, but technically this works too: during installation install on all three devices, configured as RaidZ.
  • all three disks are bootable!
  • you max out overall capacity
  • you get all the goodies of ZFS
  • (only) one device may fail
You pay for it with slower operation - half the speed of a mirror.

And even though you do not want to hear it: all Solid-State devices used for this kind of use case should be "Enterprise Class" with PLP.


Disclaimer: I do not use RaidZ for PVE anywhere = not tested by me; (nearly) all my SSDs have PLP; none of my NVMe has PLP - in my Homelab!
 
In such a case, how would you proceed to reinstall PVE so as not to lose your data?
Is it even possible?
 
gusto said:
I will install proxmox on an NVMe disk (EXT4).
The other 2 NVMe disks will serve as data storage for LXC/KVM, e.g. nextcloud data, plex data, etc ...
These 2 disks will be a ZFS Raid1 mirror.
In case of reinstalling Proxmox VE, the data on the disks with ZFS will not be corrupted.
Click to expand...
gusto said:
I even considered just 2 disks, but when reinstalling Proxmox VE I would always lose data.
Click to expand...
gusto said:
If the disk where the data will be fails, I will lose all the data.
Click to expand...
gusto said:
In such a case, how would you proceed to reinstall PVE so as not to lose your data?
Click to expand...

This all sounds like you are not even planning to have (frequent) backups at all?
 
  • Like
Reactions: Johannes S
So,
1x would be the system disk. There is no data on the system disk, but the system.
If the system disk fails, I can reinstall PVE at any time
The other two disks would have data (ZFS RAID1).

I have several independent PVE installations on different branches. Somewhere I have 4 disks (which is completely OK), but I have a branch where I have 1 SSD (ext4) and everything works properly.
Of course, I regularly back up LXC and KVM and encrypt with rclone on gdrive.

Yes, ideally there would be 4 disks, but mini PCs mostly have 2 or 3 NVMe slots (I wrote about that above).
 
If the zfs disk is not related to the Proxmox VE system disk, then the virtual disk is not lost and everything does not have to start all over again.

If you do not consider the possibility of zfs corruption in a lab environment, it would be a good idea to back up the following.
We can do the same in the configuration file of the reinstalled Proxmox

/etc/pve/qemu-server/*
/etc/network/interfaces
/etc/pve/storage.cfg
/var/lib/vz/snippets/*
/etc/modprobe.d/*
/etc/kernel/cmdline
/etc/default/grub
/usr/share/kvm/*
/etc/modules

I don't know if you need to run zfs import.
 
Last edited:
That is if you want to use a total of three nvme's, one for the system disk and two for zfs
 
You must log in or register to reply here.
Top Bottom