CEPH: Can we run 2x Cluster network subnet simultanously ?

D

DC-CA1

Member
Proxmox Subscriber
Jan 16, 2022
195
8
23
39
hi in case of a NIC or switch failure , is i possible to define 2 network ?
( we use bonding no worry , but in case also of LACP issues )

example
cluster_network = 192.168.1.0/24 192.168.2.0/24
public_network = 192.168.4.0/24 192.168.3.0/24

other question , per my understanding Monitor will only listen to 1 adress at a time right ? so it will not help in this case but for cluster ?
 
The OSDs will also only listen to one IP address.

It is possible to configure multiple IP networks for public and cluster network. This is for the case when your cluster nodes are in separate buildings and routing has to be used between them, not for failover.
 
I used the configuration with multiple cluster and public networks when I needed to migrate my network to new IP subnets. My understanding is that those two settings act like ACL by allowing connections only with the source IP within the specified range. And I believe OSD/MON select the first interface from this range to listen on. I don't think it will do any failover for you.

If you need the failover between interfaces/switches you can create a configuration with a /32 IP address on a dummy interface and the ECMP routing configuration to the physical interfaces, and have your ceph using those IP addresses. It's a more complex configuration, but works beautifully
 
  • Like
Reactions: gurubert
mfed said:
I used the configuration with multiple cluster and public networks when I needed to migrate my network to new IP subnets. My understanding is that those two settings act like ACL by allowing connections only with the source IP within the specified range. And I believe OSD/MON select the first interface from this range to listen on. I don't think it will do any failover for you.

If you need the failover between interfaces/switches you can create a configuration with a /32 IP address on a dummy interface and the ECMP routing configuration to the physical interfaces, and have your ceph using those IP addresses. It's a more complex configuration, but works beautifully
Click to expand...
Hi mfed, could you possibly give an example of the ECMP config please
TIA
 
You must log in or register to reply here.
Top Bottom