Advice Required for Best Practice Configuration of cluster disk drives

[ctrl_alt_delete]

I have 3 hosts (HP EliteDesk Mini i7-9700 each has 2 x nVME slots or can have 1 x nVME and 1 x SSD) which I have placed in Cluster. These machines were Windows 11 previously and I had to change some bios settings to boot and cancel some windows specific security.

The attached image shows the drives on all the hosts.

cobalt has 40 Gb Memory (1 x 32GB, 1 x 8Gb); 2 x 512Gb nVME
diamond has 16GB Memory (1 x 16GB); 1 x 512Gb nVME
platinum has 40GB Memory (1 x 32GB, 1 x 8Gb); 1 x 512Gb nVME

I also have a spare 1Tb nVME.

This is a new set up which will be used to run development environment for Apache Big Data Software (Hadoop, Impala, Iceberg etc); Postgres; Redis; Possibly Kubernetes, web hosting).

My assumption is that the basic install does not format all drives, it only formats the boot drive in the host.

I've also got connectivity to my iSCI NAS and the SMB NAS (btfrs) - Lots of Tb available here. Network is on Gb (Not 10Gb or 2.5Gb)

-----------------------------

Questions

1. Does the install completely clear the Disks?
2. What disk format should I use for the above use case? (ZFS; LVM; CEPH)?
3. How can I clean up the disks if there is remittance of Windows there?
4. To make each host have the same disk space in each, will I have any issues if I reconfigure the hardware such that 2 x hosts have 2 x 512gb and the other has 1Tb.

I am happy to reinstall the hosts and reconfigure the cluster again after the changes and I hope I have provided enough information.

What are your recommendations for this based on the above?

 

[Netwerkfix]

Hi ctrl_alt_delete,

The best way to deploy a Proxmox Cluster is, 3 or more hosts if you got only 1 nvme some off these hosts.
Recomended deployment is to use a ZFS pool instead off what you got now is a LVM based.

If you cluster it again you should set Replication on a VM or LXC container, there hes copying 15min=default to the other hosts raw image.
if one off these nodes fell down or loss connection other host start from that raw image and like 5min your system VM/LXC is up and running.

If you had some more questions ask them.

 

[bbgeek17]

1. Does the install completely clear the Disks?

only the boot disk, it does not touch other disks unless you really make it via advanced install

2. What disk format should I use for the above use case? (ZFS; LVM; CEPH)?

I suspect those drives are consumer type, you should not use ZFS or Ceph on them long term. Short term - feel free to experiment with any of them.

How can I clean up the disks if there is remittance of Windows there?

You can use "wipefs" (man wipefs)

To make each host have the same disk space in each, will I have any issues if I reconfigure the hardware such that 2 x hosts have 2 x 512gb and the other has 1Tb.

My question is why? Disks of different sizes have different geometries, 2x512GB is not identical to 1TB physically. Logically - sure go for it.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[ctrl_alt_delete]

Thanks for all the great advice , I can see now that if I want to set this up correctly for long term, low maintenance use, I will need to upgrade the non enterprise drives in each.

Given the network is standard Gigabit for the cluster and assuming I have to replace all the drives, what would a good optimal cluster set up be?

The 3 hosts (HP EliteDesk Mini i7-9700 each has 2 x nVME slots or can have 1 x nVME and 1 x SSD)

Use is Development only (Apache Big Data Software (Hadoop, Impala, Kudu, Iceberg etc); Postgres; Redis; Possibly Kubernetes, web hosting), a Minecraft server for the kids to connect to.

I have lots of Network storage via Synology NAS.

Given my use case above what would you experts do ?

What are good low cost enterprise nvmes?