Hello,
I have a 3 proxmox nodes cluster with ceph using an InfiniBand (IB) mesh network. I used the method 2 from this tuto to configure it since broadcast is not supported by IB.
Today, I decided to add a 4th node but I don't manage to correctly configure my network to access the ceph network from the 4th node using the communication network. Any suggestion would be great (my cluster communication is 192.168.1.x using a vlan bond, a vlan can be configure, I was thinking to bridge a vlan but in this case, I could have loop problem).
Thank you in advance for your help,
Best regards,
Rudy
First interfaces for node 1
I have a 3 proxmox nodes cluster with ceph using an InfiniBand (IB) mesh network. I used the method 2 from this tuto to configure it since broadcast is not supported by IB.
Today, I decided to add a 4th node but I don't manage to correctly configure my network to access the ceph network from the 4th node using the communication network. Any suggestion would be great (my cluster communication is 192.168.1.x using a vlan bond, a vlan can be configure, I was thinking to bridge a vlan but in this case, I could have loop problem).
Thank you in advance for your help,
Best regards,
Rudy
First interfaces for node 1
Code:
auto lo
iface lo inet loopback
iface eth0 inet manual
iface eth1 inet manual
auto ib0
iface ib0 inet static
address 10.0.1.11
netmask 255.255.255.0
pre-up modprobe ib_ipoib
pre-up echo connected > /sys/class/net/ib0/mode
mtu 65520
up ip route add 10.0.1.12/32 dev ib0
down ip route del 10.0.1.12/32
auto ib1
iface ib1 inet static
address 10.0.1.11
netmask 255.255.255.0
pre-up modprobe ib_ipoib
pre-up echo connected > /sys/class/net/ib1/mode
mtu 65520
up ip route add 10.0.1.13/32 dev ib1
down ip route del 10.0.1.13/32
auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
bond_mode 802.3ad
bond_xmit_hash_policy layer2+3
auto vmbr0
iface vmbr0 inet static
address 192.168.1.11
netmask 255.255.255.0
gateway 192.168.1.250
bridge_ports bond0.1
bridge_stp off
bridge_fd 0