Hi All,
I'd really appreciate any advice on the following issue (I already know it's far from ideal).
We have been asked to setup a new service at relatively short notice. This service requires four VMs or bare-metal servers requiring the following resources: (16 CPUs, 32GB RAM and 50GB disk). We have access to an existing VMware solution but this can't currently host these requirements due to hardware limitations. The only hardware we have available to set these up on are HPC compute nodes which have the following specification:
2 x AMD EPYC 7742 64-Core Processor
1TB RAM, 1TB SSD (single)
1x100GbE
We have plenty of these nodes but every one taken from the HPC service adds an extra delay to the HPC queues.
I could just install these new services as bare metal (take four nodes out of the HPC service) but as they are single disks there would be no redundancy in the system. I was therefore wondering if I could take three nodes out of the cluster and set them up as hyperconverged proxmox with ceph running underneath. The nodes do only have single disks so the OS would need to share with the ceph storage, and there are only single network links (so there's no traffic separation) but they are 100GbE. Also if a node fails I can at least can pull another node out of the cluster to replace it quickly.
If this is a really, really bad idea please let me know. If I can just about get away with it but there are things that I can do to minimise issues (there's plenty of RAM so perhaps make the ceph cache large if I use bluestore) then I'd also appreciate any advice. We do have a large GPFS storage system which is accessible over NFS but usage patterns on this storage means it's not suitable as a VM store (can be used for backups).
As ever this is only supposed to be temporary so will probably need to run for years!
Many thanks for any help!
I'd really appreciate any advice on the following issue (I already know it's far from ideal).
We have been asked to setup a new service at relatively short notice. This service requires four VMs or bare-metal servers requiring the following resources: (16 CPUs, 32GB RAM and 50GB disk). We have access to an existing VMware solution but this can't currently host these requirements due to hardware limitations. The only hardware we have available to set these up on are HPC compute nodes which have the following specification:
2 x AMD EPYC 7742 64-Core Processor
1TB RAM, 1TB SSD (single)
1x100GbE
We have plenty of these nodes but every one taken from the HPC service adds an extra delay to the HPC queues.
I could just install these new services as bare metal (take four nodes out of the HPC service) but as they are single disks there would be no redundancy in the system. I was therefore wondering if I could take three nodes out of the cluster and set them up as hyperconverged proxmox with ceph running underneath. The nodes do only have single disks so the OS would need to share with the ceph storage, and there are only single network links (so there's no traffic separation) but they are 100GbE. Also if a node fails I can at least can pull another node out of the cluster to replace it quickly.
If this is a really, really bad idea please let me know. If I can just about get away with it but there are things that I can do to minimise issues (there's plenty of RAM so perhaps make the ceph cache large if I use bluestore) then I'd also appreciate any advice. We do have a large GPFS storage system which is accessible over NFS but usage patterns on this storage means it's not suitable as a VM store (can be used for backups).
As ever this is only supposed to be temporary so will probably need to run for years!
Many thanks for any help!
