ProxMox 4.4 - cluster on OVH servers in vRack

[Okoolarnik]

Hello.

I have 2 serwers (SP-64) from OVH in their vRack, each with ProxMox 4.4 OVH's distribution installed.
Adding linux container is not a problem, containers are connected to the "outside world", everything works just fine, except...

I created new cluster on the first server, and I try to add a second node to it, but
pvecm add IP.ADD.RE.SS fails never finishing on "waiting for quorum...". The new node is being added to control panel of the first server but it's inactive (when trying to see the second node's view the alert shows "ssl3_get_server_certificate: certificate verify failed (596)"). "pvecm status" on neither of the servers shows the other one in the cluster.

I figured that I needed to configure VLAN in vRack, but failed to do that despite many trials. Below is the standard network configuration from the working server:

# The loopback network interface
auto lo
iface lo inet loopback

# for Routing
auto vmbr1
iface vmbr1 inet manual
        post-up /etc/pve/kvm-networking.sh
        bridge_ports dummy0
        bridge_stp off
        bridge_fd 0


# vmbr0: Bridging. Make sure to use only MAC adresses that were assigned to you.
auto vmbr0
iface vmbr0 inet static
        address 217.182.192.155
        netmask 255.255.255.0
        network 217.182.192.0
        broadcast 217.182.192.255
        gateway 217.182.192.254
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

iface vmbr0 inet6 static
        address 2001:41d0:0303:199b::
        netmask 64
        post-up /sbin/ip -f inet6 route add 2001:41d0:0303:19ff:ff:ff:ff:ff dev vmbr0
        post-up /sbin/ip -f inet6 route add default via 2001:41d0:0303:19ff:ff:ff:ff:ff
        pre-down /sbin/ip -f inet6 route del default via 2001:41d0:0303:19ff:ff:ff:ff:ff
        pre-down /sbin/ip -f inet6 route del 2001:41d0:0303:19ff:ff:ff:ff:ff dev vmbr0

Can anyone help me to configure this, so the second node could be connected to the cluster correcly?

 

[de Thysebaert]

I'll try to respons with my own experience on OVH infra.

First detect the interface connected to vRack using

#ifconfig -a | grep eth | awk '{print $1}'

for me eth0 is on public network and eth1 on the private network (VRack). (In the manager you must also link your servers with the vrack)

Second configure the /etc/network.interfaces with a private subnet , for me here are my config

#Bridge to Ovh Vrack
auto vmbr2
iface vmbr2 inet static
    address  172.16.0.1
    netmask  255.255.0.0
    network 172.16.255.255
    bridge_ports eth1
    bridge_stp off
    bridge_fd 0
    echo 1 > /proc/sys/net/ipv4/ip_forward

I also found on a forum that you must to change de hosts file to force the 2 nodes to use private network to communicate especially for the cluster

My change is made with this example

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1   localhost.localdomain localhost
<PUBLIC-IP>   FQDN  of the server
172.16.0.1   SHORT-NAME of the server
# The following lines are desirable for IPv6 capable hosts
#(added automatically by netbase upgrade)
::1     ip6-localhost ip6-loopback
.......

fr

 

[Okoolarnik]

Thank You very, very much, It worked. I had to modify a little bit /etc/network/interfaces, but Yours put me on a right track.

 

[de Thysebaert]

Suffit m'y son knowledge what change hadj you made ?

 

[Okoolarnik]

I had to add two lines after vmbr2 definition

#Bridge to Ovh Vrack
auto vmbr2
iface vmbr2 inet static
    address  172.16.0.1
    netmask  255.240.0.0
    network 172.31.255.255
    bridge_ports eth1
    bridge_stp off
    bridge_fd 0
    post-up /sbin/ip addr add 172.16.0.100/12 dev vmbr2
    post-down /sbin/ip addr del 172.16.0.100/12 dev vmbr2

Of course as You mentioned (which I also had googled) there has to be the change in /etc/hosts