PVE 4 two nodes cluster wihout HA possible ?

H

hugosxm

Member
Jul 23, 2014
46
0
6
Montpellier, France
Hi !

I am understanding that we cannot do HA with only two nodes in PVE4

But can i create a cluster to have unified management of the two hypervisor ? Or it is not possible because i will not have quorum and backup will fail etc etc ... ?

I don't want to enable HA, just create a basic cluster...

Thanks by advance :)
 
i followed the wiki

created the cluster on the first node then when i add the second one :

Code: 
Are you sure you want to continue connecting (yes/no)? yes
root@172.20.50.254's password:
copy corosync auth key
stopping pve-cluster service
backup old database
Job for corosync.service failed. See 'systemctl status corosync.service' and 'journalctl -xn' for details.
waiting for quorum...


mulitcast is working because i was in cluster with pve 3.4 on the same switch
 
on primary node

Code: 
root@proxmox /home/admin # systemctl status corosync
● corosync.service - Corosync Cluster Engine
   Loaded: loaded (/lib/systemd/system/corosync.service; enabled)
   Active: active (running) since lun. 2015-10-26 10:14:48 CET; 25min ago
  Process: 12004 ExecStart=/usr/share/corosync/corosync start (code=exited, status=0/SUCCESS)
 Main PID: 12010 (corosync)
   CGroup: /system.slice/corosync.service
           └─12010 corosync


oct. 26 10:14:48 proxmox corosync[12010]: [SERV  ] Service engine loaded: corosync cluster quorum service v0.1 [3]
oct. 26 10:14:48 proxmox corosync[12010]: [QB    ] server name: quorum
oct. 26 10:14:48 proxmox corosync[12010]: [TOTEM ] A new membership (172.20.50.254:4) was formed. Members joined: 1
oct. 26 10:14:48 proxmox corosync[12010]: [QUORUM] Members[1]: 1
oct. 26 10:14:48 proxmox corosync[12010]: [MAIN  ] Completed service synchronization, ready to provide service.
oct. 26 10:14:48 proxmox corosync[12004]: Starting Corosync Cluster Engine (corosync): [  OK  ]
oct. 26 10:14:48 proxmox systemd[1]: Started Corosync Cluster Engine.
oct. 26 10:17:23 proxmox corosync[12010]: [CFG   ] Config reload requested by node 1
oct. 26 10:17:23 proxmox corosync[12010]: [QUORUM] This node is within the non-primary component and will NOT provide any services.
oct. 26 10:17:23 proxmox corosync[12010]: [QUORUM] Members[1]: 1


on second node that is still waiting for quorum

Code: 
root@proxmox2:~# systemctl status corosync
● corosync.service - Corosync Cluster Engine
   Loaded: loaded (/lib/systemd/system/corosync.service; enabled)
   Active: failed (Result: exit-code) since lun. 2015-10-26 10:18:35 CET; 23min ago
  Process: 19969 ExecStart=/usr/share/corosync/corosync start (code=exited, status=1/FAILURE)


oct. 26 10:17:34 proxmox2 corosync[19975]: [MAIN  ] Corosync Cluster Engine ('2.3.5'): started and ready to provide service.
oct. 26 10:17:34 proxmox2 corosync[19975]: [MAIN  ] Corosync built-in features: augeas systemd pie relro bindnow
oct. 26 10:17:34 proxmox2 corosync[19977]: [TOTEM ] Initializing transport (UDP/IP Multicast).
oct. 26 10:17:34 proxmox2 corosync[19977]: [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1
oct. 26 10:17:34 proxmox2 corosync[19977]: [TOTEM ] The network interface [172.20.50.128] is now up.
oct. 26 10:17:34 proxmox2 corosync[19977]: [SERV  ] Service engine loaded: corosync configuration map access [0]
oct. 26 10:17:34 proxmox2 corosync[19977]: [QB    ] server name: cmap
oct. 26 10:17:34 proxmox2 corosync[19977]: [SERV  ] Service engine loaded: corosync configuration service [1]
oct. 26 10:17:34 proxmox2 corosync[19977]: [QB    ] server name: cfg
oct. 26 10:17:34 proxmox2 corosync[19977]: [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
oct. 26 10:17:34 proxmox2 corosync[19977]: [QB    ] server name: cpg
oct. 26 10:17:34 proxmox2 corosync[19977]: [SERV  ] Service engine loaded: corosync profile loading service [4]
oct. 26 10:18:35 proxmox2 corosync[19969]: Starting Corosync Cluster Engine (corosync): [FAILED]
oct. 26 10:18:35 proxmox2 systemd[1]: corosync.service: control process exited, code=exited status=1
oct. 26 10:18:35 proxmox2 systemd[1]: Failed to start Corosync Cluster Engine.
oct. 26 10:18:35 proxmox2 systemd[1]: Unit corosync.service entered failed state.
 
I have the same setup and worked this out just yesterday - Try this:

man votequorum


pvecm status


nano /etc/pve/corosync.conf


quorum {
provider: corosync_votequorum
two_node: 1
wait_for_all: 0
}

totem {
config_version: INCREASE_BY_1 !!
interface {
bindnetaddr: <ensure set to network address not node address, e.g. 192.168.0.0 >
}
}



(Change takes place straight away!)


pvecm status

(check on other node too!)


Jason

 
Jason Calvert said:
I have the same setup and worked this out just yesterday - Try this:man votequorumpvecm statusnano /etc/pve/corosync.conf quorum { provider: corosync_votequorum two_node: 1 wait_for_all: 0 } totem { config_version: INCREASE_BY_1 !! interface { bindnetaddr: } }(Change takes place straight away!) pvecm status(check on other node too!)Jason
Click to expand...
Thnaks for the help but the real problem is that corosync does not want to start on node 2The log is not showing helpfull information :/


PLease see what i have just found on the node 2 journal :

Code: 
[FONT=arial]Starting Corosync Cluster Engine...[/FONT]
[FONT=arial]oct. 28 10:13:06 proxmox2 corosync[11907]: [MAIN  ] Corosync Cluster Engine ('2.3.5'): started and ready to provide service.[/FONT]
[FONT=arial]oct. 28 10:13:06 proxmox2 corosync[11907]: [MAIN  ] Corosync built-in features: augeas systemd pie relro bindnow[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [TOTEM ] Initializing transport (UDP/IP Multicast).[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha1[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 pmxcfs[1199]: [status] crit: cpg_send_message failed: 9[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 pmxcfs[1199]: [status] crit: cpg_send_message failed: 9[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 pmxcfs[1199]: [status] crit: cpg_send_message failed: 9[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 pmxcfs[1199]: [status] crit: cpg_send_message failed: 9[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 pmxcfs[1199]: [status] crit: cpg_send_message failed: 9[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 pmxcfs[1199]: [status] crit: cpg_send_message failed: 9[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [TOTEM ] The network interface [172.20.50.128] is now up.[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [SERV  ] Service engine loaded: corosync configuration map access [0][/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [QB    ] server name: cmap[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [SERV  ] Service engine loaded: corosync configuration service [1][/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [QB    ] server name: cfg[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2][/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [QB    ] server name: cpg[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [SERV  ] Service engine loaded: corosync profile loading service [4][/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [QUORUM] Using quorum provider corosync_votequorum[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [QUORUM] Quorum provider: corosync_votequorum failed to initialize.[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [SERV  ] Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quorum.expected_votes must be configured!'[/FONT]
[FONT=arial]oct. 28 10:13:07 proxmox2 corosync[11908]: [MAIN  ] Corosync Cluster Engine exiting with status 20 at service.c:356[/FONT]
 
Last edited:
On the node where corosync doesn't works, can you try the following:
Code: 
systemctl stop pve-cluster
systemctl stop corosync

# now copy the working corosync config (should have two nodes defined) to the broken node.
# from /etc/pve/corosync.conf (working node) to /etc/corosync/corosync.conf (broken node) note the different folders!

systemctl start corosync
systemctl start pve-cluster
 
t.lamprecht said:
On the node where corosync doesn't works, can you try the following:
Code: 
systemctl stop pve-clustersystemctl stop corosync# now copy the working corosync config (should have two nodes defined) to the broken node.# from /etc/pve/corosync.conf (working node) to /etc/corosync/corosync.conf (broken node) note the different folders!systemctl start corosyncsystemctl start pve-cluster
Click to expand...
i did what you said, now i have this in the log : oct. 28 15:25:09 proxmox2 pve-ha-crm[2067]: ipcc_send_rec failed: Connexion refuséeoct. 28 15:25:09 proxmox2 pve-ha-crm[2067]: ipcc_send_rec failed: Connexion refuséeoct. 28 15:25:09 proxmox2 pve-ha-crm[2067]: ipcc_send_rec failed: Connexion refuséeconnection refused :/i have just realised that i have to type the password to login on proxmox2 in ssh from proxmox1, is this normal ? maybe the key exchange has failed ?!
 
up !

i am still trying to figuring it out :/

i want to modify the corosync.conf in the second node

Code: 
 cat /etc/pve/corosync.conf
logging {
  debug: off
  to_syslog: yes
}


nodelist {
  node {
    name: proxmox
    nodeid: 1
    quorum_votes: 1
    ring0_addr: proxmox
  }


  node {
    name: proxmox2
    nodeid: 2
    quorum_votes: 1
    ring0_addr: proxmox2
  }


}


quorum {
  provider: corosync_votequorum
}


totem {
  cluster_name: pmxc
  config_version: 2
  ip_version: ipv4
  secauth: on
  version: 2
  interface {
    bindnetaddr: 172.20.50.254
    ringnumber: 0
  }


}


as you can see this is not the good settings but /etc/pve is read only... because of no quorum... so how can i modify the file ?
 
ys i know but this is not working :/

Code: 
root@proxmox2:~# pvecm status
Cannot initialize CMAP service
root@proxmox2:~# pvecm expected 1
Cannot initialize CMAP service


here is more info :

Code: 
nov. 01 20:27:14 proxmox2 corosync[1624]: [SERV  ] Service engine loaded: corosync configuration map access [0]
nov. 01 20:27:14 proxmox2 corosync[1624]: [QB    ] server name: cmap
nov. 01 20:27:14 proxmox2 corosync[1624]: [SERV  ] Service engine loaded: corosync configuration service [1]
nov. 01 20:27:14 proxmox2 corosync[1624]: [QB    ] server name: cfg
nov. 01 20:27:14 proxmox2 corosync[1624]: [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
nov. 01 20:27:14 proxmox2 corosync[1624]: [QB    ] server name: cpg
nov. 01 20:28:15 proxmox2 corosync[1594]: Starting Corosync Cluster Engine (corosync): [FAILED]
nov. 01 20:28:15 proxmox2 systemd[1]: corosync.service: control process exited, code=exited status=1
nov. 01 20:28:15 proxmox2 systemd[1]: Failed to start Corosync Cluster Engine.
nov. 01 20:28:15 proxmox2 systemd[1]: Unit corosync.service entered failed state.
root@proxmox2:~# systemctl status -l pve-cluster
● pve-cluster.service - The Proxmox VE cluster filesystem
   Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled)
   Active: active (running) since dim. 2015-11-01 20:27:13 CET; 9min ago
  Process: 1217 ExecStartPost=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
  Process: 1205 ExecStart=/usr/bin/pmxcfs $DAEMON_OPTS (code=exited, status=0/SUCCESS)
 Main PID: 1210 (pmxcfs)
   CGroup: /system.slice/pve-cluster.service
           └─1210 /usr/bin/pmxcfs


nov. 01 20:36:55 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:36:55 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:36:55 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:36:55 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:37:05 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:37:05 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:37:05 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:37:05 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:37:05 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9
nov. 01 20:37:05 proxmox2 pmxcfs[1210]: [status] crit: cpg_send_message failed: 9


tell me what you need to understand
 
Last edited:
I have reinstalled proxmox on the two nodes, and when i create the cluster on the first one, corosync does not want to start... the only thing that corosync can tell me is that it is exiting with code 1... How can i try to solve this ?! maybe there is a debug mode ?
 
This is quite late, but I've been facing this kind of problem on a network with both private and public networks. The problem was that I used the internal network for the pvecm add, and the public address was used to resolve the hosts names that were set up in corosync.conf. Just adding the correct addresses in /etc/hosts solved the problem for me.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back