Hello PVE community,
I have used PVE on a single host now for quite a while and really like it a lot, so I want to take a next step and setup a small cluster for production use in a small company. I'm considering setting it up with a Ceph filesystem though I'm not very familiar with Ceph yet tbh more like still confused a lot. Right now in the progress to setup a Ceph storage in VMs for testing purpose and to get used to it. So one question would you recommend to use Ceph in that case or should I better go another route? I would really like to have live migration and failover.
For the setup I was going to use the following hardware (3 nodes)
X11SDV-8C-TP8F (Xeon D-2146NT 8C / 16T)
64 or 128GB RAM
1x Samsung SM963 or PM981 NVMe 480 or 960GB
4x Samsung SM863a 960GB
Planning to use the NVMe as boot drive but could also use it otherwise and go for a SATA DOM or mSATA SSD as system/boot drive. Any suggestions there?
Further I thought about taking 1 or 2 of the SM863a as local storage/RAID-1 and 2 / 3 as Ceph OSDs but could change there too, just can't add more storage into the systems because I can only use short rackmount cases, instead I could later on add more nodes if needed.
I don't have VMs and CTs that are extremely resource hungry, it will probably end up being like 1 Windows VM and about 10-20 CTs, some (like 4-5) of them a little bit demanding but really nothing too crazy.
Considering I'm going to use Ceph will my setup be fine for that or should I change something there like, go for more CPU or go for more RAM? I have no idea how to estimate how much of my resources will be regularly used by Ceph and how much I have available for my VMs / CTs.
Would be happy if some more experienced users could give me some ideas and suggestions about this setup and my plans, so I don't run into too much trouble.
Also could anyone suggest a good and not too pricey 10G switch ... is Ubiquiti any good or should I better get something else?
Thanks and have a nice day,
redgoldgreen
I have used PVE on a single host now for quite a while and really like it a lot, so I want to take a next step and setup a small cluster for production use in a small company. I'm considering setting it up with a Ceph filesystem though I'm not very familiar with Ceph yet tbh more like still confused a lot. Right now in the progress to setup a Ceph storage in VMs for testing purpose and to get used to it. So one question would you recommend to use Ceph in that case or should I better go another route? I would really like to have live migration and failover.
For the setup I was going to use the following hardware (3 nodes)
X11SDV-8C-TP8F (Xeon D-2146NT 8C / 16T)
64 or 128GB RAM
1x Samsung SM963 or PM981 NVMe 480 or 960GB
4x Samsung SM863a 960GB
Planning to use the NVMe as boot drive but could also use it otherwise and go for a SATA DOM or mSATA SSD as system/boot drive. Any suggestions there?
Further I thought about taking 1 or 2 of the SM863a as local storage/RAID-1 and 2 / 3 as Ceph OSDs but could change there too, just can't add more storage into the systems because I can only use short rackmount cases, instead I could later on add more nodes if needed.
I don't have VMs and CTs that are extremely resource hungry, it will probably end up being like 1 Windows VM and about 10-20 CTs, some (like 4-5) of them a little bit demanding but really nothing too crazy.
Considering I'm going to use Ceph will my setup be fine for that or should I change something there like, go for more CPU or go for more RAM? I have no idea how to estimate how much of my resources will be regularly used by Ceph and how much I have available for my VMs / CTs.
Would be happy if some more experienced users could give me some ideas and suggestions about this setup and my plans, so I don't run into too much trouble.
Also could anyone suggest a good and not too pricey 10G switch ... is Ubiquiti any good or should I better get something else?
Thanks and have a nice day,
redgoldgreen
