Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a switch

K

klose

New Member
May 23, 2014
29
0
1
Hi everybody,


has someone tried this network configuration?

Any idea about a configuration solution without getting collisions and packet loss?

Prox1 ---- Prox2
............\.../
.............\ /
.........Prox3

The network is just for the CEPH traffic, nothing else. - This will be a test for doing
the same with dual 10 Gig on board devices. Or is there another way using 12 (3*4)
SSD (or more) without a space consuming and expensive 10 Gig Switch?

Triangle-Lan.jpg

Thanks,
Klaus
 

Attachments

  • Triangle-Lan.jpg
    Triangle-Lan.jpg
    52.5 KB · Views: 13
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

I can't see why it should not work since this is a psychical implementation of a Linux Bridge, or in network terms a hub.
 
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

Just a question : How will you do when you will want to migrate, change one off these servers or just add one ?
 
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

Three bridges in this case will get a circle.
 
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

But when you will disconnect first of the three cable, (prox1 to prox2 for example), prox 1 will not see prox2, but prox3 will see prox1 + prox2 so how the datas will be replicate ?
Unless you stop all servers or ceph services when you will want to change something

I'm wrong ?
 
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

@How will you do when you will want to migrate, change one off these servers:
- Connect the new to the existing two 1Gig switches with the other three devices.
- Set up a fourth Proxmox Server, but no CEPH on this device at this moment.
- Migrate the VMs from the one you want to change.
- Power of the old one for ever and remove the device.
- Start a CEPH Server on the new device and restore the data from the other two.

@...or just add one:
- never

Three server is the minimum for a very performant HA configuration.
You get more speed and HA for less money, but no flexibility to scale,
but this shouldn't be an issue in some use cases.

The other nic's are connected redundant to two cheap gigabit switches.
You don't spend 2x 1000$ in two 10Gig redundant CEPH Switches.
No Switch is faster, can't break, consume no power / space in the rack.
 
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

klose said:
@How will you do when you will want to migrate, change one off these servers:
- Connect the new to the existing two 1Gig switches with the other three devices.
- Set up a fourth Proxmox Server, but no CEPH on this device at this moment.
- Migrate the VMs from the one you want to change.
- Power of the old one for ever and remove the device.
- Start a CEPH Server on the new device and restore the data from the other two.

@...or just add one:
- never

Three server is the minimum for a very performant HA configuration.
You get more speed and HA for less money, but no flexibility to scale,
but this shouldn't be an issue in some use cases.

The other nic's are connected redundant to two cheap gigabit switches.
You don't spend 2x 1000$ in two 10Gig redundant CEPH Switches.
No Switch is faster, can't break, consume no power / space in the rack.
Click to expand...


And if one of the 6 10Gig network card fall ?
 
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

That's the point. - I asked for:
"Any idea about a configuration solution without getting collisions and packet loss?"

You can shut down one CEPH Server (for instance for changing the server itself),
but the configuration (to find) of the network should make this possible:

Loose one nic or lan cable and all three server are still up and running. Only the
traffic of the lost connection will go in an active-backup scenario over the two still
working connections. - The load is now higher, but you get redundancy for free.
Exactly it is still 5 Gig /sec from every instance to every other instance.

Not to bad for a worst case scenario without any costs.
 
Last edited:
Re: Network just for the CEPH traffic of three hosts in a "Triangle-Lan" without a sw

@"i don't know how it react when one of three node is seen by only one of the other."

When this should happen? - When you really loose one server the other see each
other with a wight of 2. That is important when there is suddenly the CEPH Server
you switched in the configuration, the third one.
 
You must log in or register to reply here.
Top Bottom