first-time setup

[me_]

Hi!

running two linux server for around eight years, I'd like to learn and migrate to proxmox. Help with the initial setup is highly appreciated. PVE 8.4 is running. Before continuing I am primarily concerned with questions such as HW-RAID-Controller yes/no, storage pool setup for the VMs, and subsequently considerations regarding replication and backup.

Starting point: 2 HPE Microserver with each 4x 4TB ([1] raidz1, [2] mdadm raid5): two machines for two small 'organizations' running cloud, caldav, nginx, samba-shares, etc.
Migration to new hardware: HPE Server with 8x 8TB HDDs for storage; OS: 2x 500GB M.2 SSD SATA (ZFS RAID-1 created whilst installation of PVE 8.4)

Preliminary considerations: Without doing any further research, I bought a HPE Smart Array P408i-p SR Gen10 controller. After research, I have now connected the 8 hard disks directly via the internal SATA storage system.

In general, I plan to create two VMs for the two 'organizations', as it is the case on the actual hardware, plus any additional LXC or VMs where it makes sense from an application point of view to separate them (web server for company-related websites, etc.).

Storage: Is it advisable to divide the 8x 8TB into two ZFS pools (for the two 'organizations') or to create a ZFS-RAID from all 8 HDDs? With replication and backup in mind, I would create two raidz1 with each 4 HDDs, or a raidz2 for all 8 HDDs? storage currently:
rpool (500GB):
– local (dir.): Backup, ISO image, Container template; /var/lib/vz
– local-zfs (ZFS): Disk image, Container;

As soon as the data has been transferred from the old hardware, I would possibly set up the two remaining microserver as additional nodes for a cluster configuration. In this case, would there be anything else to consider when configuring the storage pools, except the same naming?
Still not sure, if it makes sense to run all three servers with regard to power consumption. We run a production environment, but short downtimes are okay.

Cluster: What is best practice for production environments? Are critical updates and changes to the system tested on a duplicate (live-migrate) and then imported? As I understand ZFS replication is besides High Availability (HA), useful in the event of a system failure?

Backup: Currently there are two Synology NAS (4x 4TB each) running Active Backup for Business. I would keep it that way, or possibly implement Proxmox Backup Server (PBS).

Thank you!

 

[FSIN]

Hi,

When using ZFS, you shouldn't use a hardware RAID controller. ZFS requires direct access to the disks in order to manage redundancy, error detection, and recovery. Hardware RAID abstracts the disks from the operating system, making it impossible for ZFS to function correctly. You should configure your HPE Smart Array P408i-p in HBA mode (Here is an Article).

For your 8x 8TB HDDs, you should use a single ZFS RAIDZ2 pool. This provides two-disk fault tolerance, a good balance of performance, capacity, and safety. It also simplifies management compared to two separate RAIDZ1 pools, which would only offer single disk redundancy per group and higher risk during resilvering.

For better data management, create separate datasets for each organization, such as data/org1 and data/org2. This provides logical separation and allows you to apply different properties (compression, quotas, etc.) to each.

If you are considering scaling later, and if your hosts can be connected using a fast low-latency interconnect (e.g., 10G or better), you could explore CEPH as a distributed storage backend. Note that CEPH requires at least three nodes for redundancy and stable quorum. Given your current hardware, i think ZFS is better suited.

You should configure a Proxmox cluster. Although true High Availability (HA) requires three nodes to maintain quorum and avoid split-brain issues, two nodes can still be configured for scheduled replication and manual failover. You can use one node as the primary compute node and the other as a standby or replication target. Automatic load balancing with Proxlib is possible, though it requires advanced configuration and scripting. If you later add a third node (even a lightweight quorum-only node or QDevice), you can enable proper HA with fencing.

For backup, you can continue using your Synology NAS and Active Backup for Business as an additional backup layer. However, for full integration with Proxmox, setting up a Proxmox Backup Server (PBS) is recommended. You could repurpose one of your old servers as a dedicated PBS instance and use the Synology NAS as a secondary target via scheduled PBS sync or remote copy jobs. PBS offers deduplication, encryption, and efficient incremental backups tightly integrated into Proxmox.

regards
Frederik

 

[LnxBil]

For your 8x 8TB HDDs, you should use a single ZFS RAIDZ2 pool. This provides two-disk fault tolerance, a good balance of performance, capacity, and safety.

I would not have added the word performance to that sentence with a single harddisk vdev raidz2 in that pool. I would never run such a setup in production, not even at home.

Hopefully, all disks (hard disk and SSDs) are enterprise hardware and not con/prosumer ones. If you have enterprise SSDs, I would consider running everything in one pool: stripped raid1 over all disks (4x write performance in comparison to any raidz stuff) and the two SSDs as special devices for metadata and stuff that needs to be fast. Just search the forum or look into the reference documentation.