Problem mit pvecm nodes

iTweek

Well-Known Member
Jan 2, 2017
136
4
58
Guten Abend community.
Ich habe nun ein zweiten rechner in ein Cluster einbinden wollen.

Folgendes habe ich gemacht

node1 hat schon vms am laufen. da habe ich

pvecm create YOUR-CLUSTER-NAME
pvecm status

Code:
Membership information
----------------------
    Nodeid      Votes Name
         1          1 ju325 (local)
root@ju325:~# pvecm status
Quorum information
------------------
Date:             Mon Feb  6 21:12:43 2017
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000001
Ring ID:          1/4
Quorate:          No

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      1
Quorum:           2 Activity blocked
Flags:

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 89.163.**** (local)

So beim zweiten server neue ist von der hardware nicht der selbe. habe ich gemacht.

pvecm add (ip address)

Code:
root@89.163.****'s password:
copy corosync auth key
stopping pve-cluster service
backup old database
Job for corosync.service failed. See 'systemctl status corosync.service' and 'journalctl -xn' for de                                                     tails.
waiting for quorum...

so wenn ich jetzt

pvecm nodes mache kommt -> Cannot initialize CMAP service
darauf hin hab ich systemctl status corosync.service -l gemacht

Code:
● corosync.service - Corosync Cluster Engine
   Loaded: loaded (/lib/systemd/system/corosync.service; enabled)
   Active: failed (Result: exit-code) since Mon 2017-02-06 21:22:17 CET; 15min ago
  Process: 3159 ExecStart=/usr/share/corosync/corosync start (code=exited, status=1/FAILURE)

Feb 06 21:21:16 ve1257 corosync[3166]: [SERV  ] Service engine loaded: corosync configuration service [1]
Feb 06 21:21:16 ve1257 corosync[3166]: [QB    ] server name: cfg
Feb 06 21:21:16 ve1257 corosync[3166]: [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Feb 06 21:21:16 ve1257 corosync[3166]: [QB    ] server name: cpg
Feb 06 21:21:16 ve1257 corosync[3166]: [SERV  ] Service engine loaded: corosync profile loading service [4]
Feb 06 21:21:16 ve1257 corosync[3166]: [QUORUM] Using quorum provider corosync_votequorum
Feb 06 21:22:17 ve1257 corosync[3159]: Starting Corosync Cluster Engine (corosync): [FAILED]
Feb 06 21:22:17 ve1257 systemd[1]: corosync.service: control process exited, code=exited status=1
Feb 06 21:22:17 ve1257 systemd[1]: Failed to start Corosync Cluster Engine.
Feb 06 21:22:17 ve1257 systemd[1]: Unit corosync.service entered failed state.

allerdings ist der server laut webinterface eingetragen? Der eine wird als Rot angezeigt. Ich denke irgend was ist schief gelaufen. Wie bekomme ich das behoben? bin da echt ratlos ohne das ganze system jetzt zu zerstören.

was mir auf gefallen ist das es zwei unterschiedliche versionen sind. Laut google macht das aber nichts oder? pve1 4.3-14 pve2 4.4-12 (pve2 ist error.)

ich bin um jede hilfe dankbar^^
 
wenn ich journalctl -xn mache kommt folgendes

Code:
-- Logs begin at Mon 2017-02-06 20:52:15 CET, end at Mon 2017-02-06 21:53:58 CET. --
Feb 06 21:53:51 ve1257 pveproxy[1162]: starting 1 worker(s)
Feb 06 21:53:51 ve1257 pveproxy[1162]: worker 5472 started
Feb 06 21:53:51 ve1257 pveproxy[5472]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/
Feb 06 21:53:53 ve1257 pve-ha-lrm[1169]: unable to write lrm status file - unable to open file '/etc/pve/nodes/ve1257/lrm_status.tmp
Feb 06 21:53:56 ve1257 pveproxy[5472]: worker exit
Feb 06 21:53:56 ve1257 pveproxy[1162]: worker 5472 finished
Feb 06 21:53:56 ve1257 pveproxy[1162]: starting 1 worker(s)
Feb 06 21:53:56 ve1257 pveproxy[1162]: worker 5479 started
Feb 06 21:53:56 ve1257 pveproxy[5479]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/
Feb 06 21:53:58 ve1257 pve-ha-lrm[1169]: unable to write lrm status file - unable to open file '/etc/pve/nodes/ve1257/lrm_status.tmp
 
habe noch ein weiteres problem.. wie bekomme ich nun den Node wieder aus dem webinterface? wenn ich pvecm nodes mache dann wird der kapuute node nicht angezeigt aber im webinterface.
Ich verzweifle grade xD

[EDIT mit dem webinterface hat sich grade erledigt habe ihn wieder raus bekommen.]

Aber nun zum problem mit dem einfügen.
 
Last edited:
Danke. Habe nun mal geschaut. so weit wieder alles zurück machen können..

Jetzt wenn ich

pvecm add 89.163.251.**
root@89.163.251.**'s password:

unable to add node: command failed (ssh 89.163.251.** -o BatchMode=yes pvecm addnode ve1257 --force 1)

was mache ich nun falsch?


habe grade gesehen
pvc1 in systemlog.
hat das was damit zutun?
Feb 07 18:09:24 ju325 sshd[1105]: Connection closed by 78.31.67.**[preauth]
Feb 07 18:09:24 ju325 sshd[1107]: Authentication refused: bad ownership or modes for directory /etc


edit habe grade gesehn das der ordner falsche rechte hatte.

jetzt konnte ich das hier machen.

pvecm add 89.163.251.**
copy corosync auth key
stopping pve-cluster service
backup old database
Job for corosync.service failed. See 'systemctl status corosync.service' and 'journalctl -xn' for details.
waiting for quorum...


so nun passiert nichts mehr.

Code:
● corosync.service - Corosync Cluster Engine
   Loaded: loaded (/lib/systemd/system/corosync.service; enabled)
   Active: failed (Result: exit-code) since Tue 2017-02-07 18:16:57 CET; 58s ago
  Process: 3366 ExecStart=/usr/share/corosync/corosync start (code=exited, status=1/FAILURE)

Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine loaded: corosync configuration service [1]
Feb 07 18:15:56 ve1257 corosync[3375]: [QB    ] server name: cfg
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Feb 07 18:15:56 ve1257 corosync[3375]: [QB    ] server name: cpg
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine loaded: corosync profile loading service [4]
Feb 07 18:15:56 ve1257 corosync[3375]: [QUORUM] Using quorum provider corosync_votequorum
Feb 07 18:15:56 ve1257 corosync[3375]: [QUORUM] Quorum provider: corosync_votequorum failed to initialize.
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quor...figured!'
Feb 07 18:15:56 ve1257 corosync[3375]: [MAIN  ] Corosync Cluster Engine exiting with status 20 at service.c:356.
Feb 07 18:16:57 ve1257 corosync[3366]: Starting Corosync Cluster Engine (corosync): [FAILED]
Feb 07 18:16:57 ve1257 systemd[1]: corosync.service: control process exited, code=exited status=1
Feb 07 18:16:57 ve1257 systemd[1]: Failed to start Corosync Cluster Engine.
Feb 07 18:16:57 ve1257 systemd[1]: Unit corosync.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.
jemand eine idee?
 
Last edited:
Danke für die antwort.

Wie kein Quorum? habe doch im PVE1

pvecm create YOUR-CLUSTER-NAME bzw your ersetzt durch mein name.

und dann im zweiten PVE2 dann

pvecm add <ip von pve1> gemacht?

Wenn das quorum auf pve1 fehlt wie mache ich das?


### edit

habe nun wieder weiter gekommen. Bleibe wieder hängen bei ->

root@89.163.****'s password:
copy corosync auth key
stopping pve-cluster service
backup old database
Job for corosync.service failed. See 'systemctl status corosync.service' and 'journalctl -xn' for de tails.
waiting for quorum...


Code:
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine loaded: corosync configuration service [1]
Feb 07 18:15:56 ve1257 corosync[3375]: [QB    ] server name: cfg
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Feb 07 18:15:56 ve1257 corosync[3375]: [QB    ] server name: cpg
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine loaded: corosync profile loading service [4]
Feb 07 18:15:56 ve1257 corosync[3375]: [QUORUM] Using quorum provider corosync_votequorum
Feb 07 18:15:56 ve1257 corosync[3375]: [QUORUM] Quorum provider: corosync_votequorum failed to initialize.
Feb 07 18:15:56 ve1257 corosync[3375]: [SERV  ] Service engine 'corosync_quorum' failed to load for reason 'configuration error: nodelist or quor...figured!'
Feb 07 18:15:56 ve1257 corosync[3375]: [MAIN  ] Corosync Cluster Engine exiting with status 20 at service.c:356.
Feb 07 18:16:57 ve1257 corosync[3366]: Starting Corosync Cluster Engine (corosync): [FAILED]
Feb 07 18:16:57 ve1257 systemd[1]: corosync.service: control process exited, code=exited status=1
Feb 07 18:16:57 ve1257 systemd[1]: Failed to start Corosync Cluster Engine.
Feb 07 18:16:57 ve1257 systemd[1]: Unit corosync.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.

Code:
-- Logs begin at Tue 2017-02-07 17:45:53 CET, end at Tue 2017-02-07 18:24:39 CET. --
Feb 07 18:24:34 ve1257 pveproxy[4496]: /etc/pve/local/pve-ssl.key: failed to load loc
Feb 07 18:24:37 ve1257 pmxcfs[3356]: [quorum] crit: quorum_initialize failed: 2
Feb 07 18:24:37 ve1257 pmxcfs[3356]: [confdb] crit: cmap_initialize failed: 2
Feb 07 18:24:37 ve1257 pmxcfs[3356]: [dcdb] crit: cpg_initialize failed: 2
Feb 07 18:24:37 ve1257 pmxcfs[3356]: [status] crit: cpg_initialize failed: 2
Feb 07 18:24:39 ve1257 pveproxy[4496]: worker exit
Feb 07 18:24:39 ve1257 pveproxy[1158]: worker 4496 finished
Feb 07 18:24:39 ve1257 pveproxy[1158]: starting 1 worker(s)
Feb 07 18:24:39 ve1257 pveproxy[1158]: worker 4508 started
Feb 07 18:24:39 ve1257 pveproxy[4508]: /etc/pve/local/pve-ssl.key: failed to load loc


ich weiß nicht mehr weiter
 
Last edited:
leigt es daran das beide server eine feste ip haben und keine vlan oder so?

denke es liegt an multicast? Kann mir wer helfen? Ich blicke es nicht.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!