2.1 Cluster and node issues

D

dns173

Active Member
Dec 12, 2011
65
0
26
hello!

i have three servers . create cluster
server1 192.168.1.31 master
server2 192.168.1.32 nodes01
server3 192.168.1.33 nodes02

i login server1 192.168.1.31 create cluster name vpshost ( 'pvecm create vpshost' ) . ok.
i login server2 and server3 create node . inpute : pvecm add 192.168.1.31 . ok create cluster finish.

but i master (server1) 192.168.1.31 the hardware is damage. i think new server create master.

i input run command : 'pvecm create vpshost' for new server (new server1 192.168.1.31) .
i think add server2 and server3 to new server1 create cluster.

but i login server2 and server3 run command :'pvecm add 192.168.1.31' error.

root@nodes02:/var/lib# pvecm add 192.168.1.60
authentication key already exists

root@nodes02:/var/lib# pvecm nodes
Node Sts Inc Joined Name
1 X 0 master
3 M 648 2012-05-26 11:35:38 nodes02
 
Last edited:
dns173 said:
hello!

i have three servers . create cluster
server1 192.168.1.31 master
server2 192.168.1.32 nodes01
server3 192.168.1.32 nodes02

i login server1 192.168.1.31 create cluster name vpshost ( 'pvecm create vpshost' ) . ok.
i login server2 and server3 create node . inpute : pvecm add 192.168.1.31 . ok create cluster finish.

but i master (server1) 192.168.1.31 the hardware is damage. i think new server create master.

i input run command : 'pvecm create vpshost' for new server (new server1 192.168.1.31) .
i think add server2 and server3 to new server1 create cluster.

but i login server2 and server3 run command :'pvecm add 192.168.1.31' error.

root@nodes02:/var/lib# pvecm add 192.168.1.60
authentication key already exists

root@nodes02:/var/lib# pvecm nodes
Node Sts Inc Joined Name
1 X 0 master
3 M 648 2012-05-26 11:35:38 nodes02
Click to expand...
Hi,
with pve2.x you don't have an master anymore...
So if your server1 is broken, server2+3 are valid clustermember with quorum. You only need to replace server1 (with an fresh install) and join them to one of the remaining cluster-server. Because the doubled name you must use "- force".

Udo
 
new server1 reinstall 2.x system use "- force"? what way join server2+3 ?
please tell me command for new server . thanks.

and old server1 (old master node) what way delete?


run command 'pvecm add 192.168.1.32 -force' 192.168.1.32(nodes01) ?? this command Right?
 
Last edited:
dns173 said:
new server1 reinstall 2.x system use "- force"? what way join server2+3 ?
please tell me command for new server . thanks.

and old server1 (old master node) what way delete?


run command 'pvecm add 192.168.1.32 -force' 192.168.1.32(nodes01) ?? this command Right?
Click to expand...
Hi,
on server2 delete first node 1 (what you called master):
Code: 
pvecm delnode server1
after that, join the new server1 to the exsisting cluster.

On server1:
Code: 
pvecm add 192.168.1.32 -force
Udo
 
I have for 2 days the same problem.


have it resolved as follows ... but believe that it is not running properly
because I have the next problem with fencing.
see also: http://forum.proxmox.com/threads/9797-Error-writing-etc-pve-cluster-conf-Permission-denied


Code: 
SERVER1:
  pvecm create CLUSTER01
 
# pvecm create CLUSTER01
Generating public/private rsa key pair.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
28:8f:da:10:2e:1b:64:a6:cf:ef:da:a7:09:6c:72:6e root@x701
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
|       .         |
| +. . . S        |
|=o . +           |
|= B . .          |
| @E* ..          |
|..*+Bo           |
+-----------------+
Restarting pve cluster filesystem: pve-cluster[dcdb] notice: wrote new cluster config '/etc/cluster/cluster.conf'
.
Starting cluster: 
   Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... [  OK  ]
   Waiting for quorum... [  OK  ]
   Starting fenced... [  OK  ]
   Starting dlm_controld... [  OK  ]
   Unfencing self... [  OK  ]
 
SERVER2:
  pvecm create CLUSTER01
 
# pvecm create CLUSTER01
Generating public/private rsa key pair.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
bf:10:f0:35:ee:ce:9e:3c:3b:58:11:8d:41:ad:f6:09 root@x702
The key's randomart image is:
+--[ RSA 2048]----+
|         .o=     |
|          o o    |
|      .   oo     |
|       o oE.     |
|        S..+ .   |
|         +. o    |
|        .oo      |
|        .=oo     |
|         .O+     |
+-----------------+
Restarting pve cluster filesystem: pve-cluster[dcdb] notice: wrote new cluster config '/etc/cluster/cluster.conf'
.
Starting cluster: 
   Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... [  OK  ]
   Waiting for quorum... [  OK  ]
   Starting fenced... [  OK  ]
   Starting dlm_controld... [  OK  ]
   Unfencing self... [  OK  ]
 
SERVER1:
  pvecm s
 
# pvecm s
Version: 6.2.0
Config Version: 1
Cluster Name: CLUSTER01
Cluster Id: 37345
Cluster Member: Yes
Cluster Generation: 4
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1  
Active subsystems: 5
Flags: 
Ports Bound: 0  
Node name: x701
Node ID: 1
Multicast addresses: 239.192.145.115 
Node addresses: 80.190.120.8
 
  cat /etc/pve/cluster.conf
 
# cat /etc/pve/cluster.conf
<?xml version="1.0"?>
<cluster name="CLUSTER01" config_version="1">
 
  <cman keyfile="/var/lib/pve-cluster/corosync.authkey">
  </cman>
 
  <clusternodes>
  <clusternode name="x701" votes="1" nodeid="1"/>
  </clusternodes>
 
</cluster>
 
SERVER2:
  pvecm s
 
# pvecm s
Version: 6.2.0
Config Version: 1
Cluster Name: CLUSTER01
Cluster Id: 37345
Cluster Member: Yes
Cluster Generation: 4
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1  
Active subsystems: 5
Flags: 
Ports Bound: 0  
Node name: x702
Node ID: 1
Multicast addresses: 239.192.145.115 
Node addresses: 80.190.120.9
 
SERVER1:
  pvecm add 80.190.120.9 [-force]
 
# pvecm add 80.190.120.9
authentication key already exists
 
  pvecm add 80.190.120.9 -force
 
# pvecm add 80.190.120.9 -force
The authenticity of host '80.190.120.9 (80.190.120.9)' can't be established.
RSA key fingerprint is 8e:72:38:a7:23:97:15:f0:37:d8:0a:d1:c1:cb:87:c7.
Are you sure you want to continue connecting (yes/no)? yes
root@80.190.120.9's password: 
copy corosync auth key
stopping pve-cluster service
Stopping pve cluster filesystem: pve-cluster.
backup old database
Starting pve cluster filesystem : pve-cluster.
Starting cluster: 
   Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... [  OK  ]
   Waiting for quorum... [  OK  ]
   Starting fenced... [  OK  ]
   Starting dlm_controld... [  OK  ]
   Unfencing self... [  OK  ]
generating node certificates
merge known_hosts file
restart services
Restarting PVE Daemon: pvedaemon.
Restarting web server: apache2 ... waiting .
successfully added node 'x701' to cluster.
 
  cat /etc/pve/cluster.conf
 
<?xml version="1.0"?>
<cluster name="CLUSTER01" config_version="2">
 
  <cman keyfile="/var/lib/pve-cluster/corosync.authkey">
  </cman>
 
  <clusternodes>
  <clusternode name="x702" votes="1" nodeid="1"/>
  <clusternode name="x701" votes="1" nodeid="2"/></clusternodes>
 
</cluster>
 
  pvecm n
 
# pvecm n
Node  Sts   Inc   Joined               Name
   1   M      4   2012-05-26 15:12:41  x701
 
 
SERVER2:
  pvecm n
 
# pvecm n
Node  Sts   Inc   Joined               Name
   1   M      4   2012-05-26 15:13:38  x702
   2   X      0                        x701
 
SERVER1 + SERVER2
  reboot
 
Those only (SERVER2 = [URL="https://x702.saverserver.eu:8006"]https://<server2>:8006[/URL]) available ... with two nodes!
 
Hi tom,

I've missed because I have problems with "add" but had "add -force" will work

Create the Cluster .... Login via ssh to the first Proxmox VE node.

OK, I Install new!
 
Last edited:
  • Like
Reactions: mcparlandj
tom said:
you run "create" on both nodes, but you should use "add" instead. see http://pve.proxmox.com/wiki/Proxmox_VE_2.0_Cluster#Adding_nodes_to_the_Cluster

you did:

> SERVER1: pvecm create CLUSTER01
> SERVER2: pvecm create CLUSTER01
Click to expand...

Hi Tom

I did the same mistake as liosl. I've been trying all day to delete cluster.conf to undo the wrong pvecm create but as /etc/pve is not allowing me to do anything and YOUR DOCUMENTATION is almost inexistent for version 2.1 at least, I'm out of options.
I can reinstall of course as this is my first try to proxmox I don't have any VM created yet, but seriously, you need to work on documentation urgently.
I'm planning to buy proxmox as an alternative to rhev because of the price, but I'm starting to have second thoughts because in this scenario I'll waste my 3 anual tickets in basic stuff.
Regards,
Juan Pablo
 
pvecm create <name> command is needed to run on all the nodes while creating the cluster.
OR only need to run on the master node.

Please reply..
 
#pvecm create <name>

this command creates the cluster, before there is one.
once created the cluster (executing that command on a node), the same node becomes the first cluster member.

to add other members, from their command line use "pvecm add" command (on an empty node, same pve version, not already part of any cluster)
that will add another node to the same cluster

pve now has multi master clusters, so any member node will act as master.

see also http://pve.proxmox.com/wiki/Proxmox_VE_2.0_Cluster (pve 3.x has some difference with 2.x but that page should work)

Marco
 
Sir,
I have the following network configurations.

auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
address 172.16.16.153
netmask 255.255.255.240
gateway 172.16.16.158
bridge_ports eth0
bridge_stp off
bridge_fd 0

iface eth1 inet manual

auto vmbr1
iface vmbr1 inet static
address 192.168.1.153
netmask 255.255.255.0
bridge_ports eth1
bridge_stp off
bridge_fd 0

after editing "iface eth1 manual" The SAN is not going to be visible there.

Could I make it "iface eth1 manual" to "iface eth1 dhcp" and create the vmbr1 ?

Please reply .
 
Last edited by a moderator:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back