Proxmox setup

[jmkrause78]

I currently am using vMware however, my hardware is no longer supported so was looking at moving to Proxmox. This is what I have below:

2 Dell r820 (1 has 4 cpus 256GB Ram and the other 2 cpus 128GB ram)
- Both have 16 900GB SAS drives, dual 10gig, 4 1gig Nics and pcie nvme 2tb

I am looking to set them up and either replicate the local storage or use Ceph with a raspberry pi as my witness node. Question is i am not familiar with how to setup my OS/Caching drive. I would like to use the 2TB NVME to install proxmox on but also partition it out where it can be used as a caching drive as well. Would also need some advice on how to setup the networking side of things. i have a 8 port mikrotik SFP+ switch as well as 2 Cisco 3750x 48 port switches with the 10gig module. An assistance would be greatly appreciated.

 

[Dominic]

I would not recommend two-node Ceph cluster. Instead you could look into ZFS & storage replication.
In addition, I would avoid the partitioning. You can give PVE one of the slower drives (or an additional 128GB SSD) and use the NVME as cache for ZFS. Note that for both Ceph & ZFS you should use a HBA instead of hardware RAID.

Concerning the networking, the first important step is to separate storage traffic (10G) from other traffic like corosync/cluster traffic (can be 1G, but latency must consistently be low). From there on you have more choices. It is, for example, easy to set up multiple links for redundancy for corosync.