I've been running PVE in a standalone server in a home environment for some time now and want to get into clustering with ceph, so I got 2 more identical boxes for a 3-node cluster. I've been reading a lot of the documentation and guides for clustering, but am getting confused on the terms being used and hoping someone can straighten things out.
Each box has 2 onboard Gbe ports from the mbd and 2x 25G ports from NIC. Currently, the standalone server is only connected with one of the 25G ports as the management interface and bridge. Since it is best practice to have separate networks, I was thinking to use the other 25G port for the ceph cluster network (it will be running in 10G since I do not have enough 25g ports) and one of the GBe ports for the corosync network. For the public network, is it supposed to be the same as the one for the management interface? Also, I read about having a migration network in the documentation, but I'm not sure where it fits in.
I'm also thinking of connecting all of them to the same switch and using different VLANs for the networks. Another option is to directly connect the 3 nodes together with the 2 Gbe ports (Node 1 connects to 2 and 3, Node 2 to 3 and 1, etc.), but I do not know how that would be configured. Please also let me know if there is a better way to do things.
Each box has 2 onboard Gbe ports from the mbd and 2x 25G ports from NIC. Currently, the standalone server is only connected with one of the 25G ports as the management interface and bridge. Since it is best practice to have separate networks, I was thinking to use the other 25G port for the ceph cluster network (it will be running in 10G since I do not have enough 25g ports) and one of the GBe ports for the corosync network. For the public network, is it supposed to be the same as the one for the management interface? Also, I read about having a migration network in the documentation, but I'm not sure where it fits in.
I'm also thinking of connecting all of them to the same switch and using different VLANs for the networks. Another option is to directly connect the 3 nodes together with the 2 Gbe ports (Node 1 connects to 2 and 3, Node 2 to 3 and 1, etc.), but I do not know how that would be configured. Please also let me know if there is a better way to do things.
