What would be good options for a high-availability cluster setup in our startup?

[Independent-Duty8682]

Hey,
We're planning our setup using Proxmox and were hoping to get some feedback as to how good of an idea is it:
So our current setup is 4 servers with wildly different specs:

• All of them have 2x Intel(R) Xeon(R)
• All of them have either 128GB or 512GB RAM
• All of them have at least one hardware RAID storage of varying specs and size
• All of them have at 2 other drives of varying specs
• One of them randomly shuts down for reasons to be inspected
• All of them reside in one rack

All of which will be running Proxmox
Our goal is hosting of:

• Internal Services (Git, Matrix)
• Legacy client projects ( Mail servers, static websites )
• New projects ( Dynamic websites, requiring databases, persistent storage etc )

We want to be resilient against failures of any hard drive, RAID or entire server. We are currently en route to deploy with Nomad and Ceph.
For the company however, it is important to offer a certain ease of use for the non-DevOps programmers - i.e. ability to deploy an app without having to know the details of the cluster. ( That was the initial reason Kubernetes was rejected, because of complicated setup of new apps ). That is not possible afaik on our current setup either .
What would be the best course of action for us in this scenario? Disregard our current path (Nomad, Ceph), if there is a easier, more functional way, we will take it.
Our questions mainly are:

• What would be the an appropriate orchestration software for our case ( we are currently split between Kubernetes and Nomad ) ?
• What would be an appropriate storage solution for our case? ( Ceph vs ZFS pools maybe? )
• What would be an appropriate DevOps automation pipeline?
  • Due to the ease-of-use by programmers, we would require to be able to deploy an app, test it, allocate storage and other resources, without the knowledge of the cluster.

Thank you