[SOLVED] Quorum: 3 Activity blocked

H

halley80

Well-Known Member
Feb 13, 2019
49
2
48
53
Hi,

My cluster consists of five nodes.
Following a sudden change in the ips vlan of my nodes, I modified the configuration files (network/interface).

For four nodes the 'corosync/pvecm status' is OK.
However, for the last node, I made the command => pvecm expected 1.

Since then, it's not recognized in the cluster but I can access it and the other VMs => cf screenshot

*******************************************
Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 1
Quorum: 3 Activity blocked
Flags:
*******************************************

Do I have to run the node join to the cluster again?
Thanking you for your answers,
Yours sincerely,
 

Attachments

  • Cluster_PVE.png
    Cluster_PVE.png
    42.7 KB · Views: 25
Now, when i do pvecm staus, i have :

root@ipmpve6:~# pvecm status
ipcc_send_rec[1] failed: Connection refused
ipcc_send_rec[2] failed: Connection refused
ipcc_send_rec[3] failed: Connection refused
Unable to load access control list: Connection refused

Thanks for your help
 
halley80 said:
Following a sudden change in the ips vlan of my nodes, I modified the configuration files (network/interface).
Click to expand...

Sudden change? How's that? I guess you did not adapted the corosync configuration(s).

halley80 said:
However, for the last node, I made the command => pvecm expected 1.
Click to expand...

Why and under what assumption did you do that?? And did you make changes to any configuration files on /etc/pve or /etc/corosync after that?

Please check and post the outputs of:
* on the node with problems systemctl status pve-cluster corosync
* post from the problematic node cat /etc/corosync/corosync.conf
* post from the working node cat /etc/pve/corosync.conf

(note the different paths)
 
Hi,

Thank you for your response.

The change of Vlan is due to a request to a pole of our university.
But it did not go well ....
I didn't change the corosync file
but just the network/interfaces for the Vlan assignment 'Tag VLAN' => Ex: enp94s0f0np0.10

It worked well on the other node.
implies but not this one.
Attach the requested files.
Thanking you,
Yours sincerely,
 

Attachments

  • corososync_ipm1.txt
    845 bytes · Views: 18
  • corososync_ipm6.txt
    845 bytes · Views: 12
  • Status.txt
    2.7 KB · Views: 13
From the information you shared, servers are not able to communicate to each other, if you have recently changed the vlan membership, please note they must be on same VLAN or reachable for cluster to work
can you please post the
cat /etc/hosts
and output of the following command

ping cluster-ipm1
ping cluster-ipm2
ping cluster-ipm3
ping cluster-ipm4
ping cluster-ipm5
ping cluster-ipm6
ping cluster-ipm7

from one of the server
 
Additionally to that check if multicast is still allowed and works for that node.

You may then, after you fixed the /etc/hosts (or changed corosync.conf to use IPs not hostnames for the ringX_addr entries):
Bash: 
systemctl reset-failed pve-cluster
systemctl restart corosync pve-cluster

on that node.
 
Hi,

Thank you for your answers.
Attached is the host file and the configuration of the ipm6 node retrieved from node 1.
The hosts file is identical on all nodes and ipm6 ping the other nodes without any problem.

In fact, I am no longer any files in the /etc/pve directory on node 6.

Do you think I can retrieve them from another node, such as ipm1?
Thanking you for your answers,
Yours sincerely,
 

Attachments

  • Config_ipm6.txt
    868 bytes · Views: 10
  • hosts.txt
    580 bytes · Views: 12
check corosync configuration, bindnetaddr must be IP not a network address
Please add the correct IP of the node and then restart the following

systemctl restart pve-cluster
systemctl restart corosync

put the output of the following command after the above

pvecm status
 
Thank you for your response but:
the /etc/pve directory is empty ....

and the ping works correctly from this node (6)
 
root@ipmpve6:~# pvecm status
ipcc_send_rec[1] failed: Connection refused
ipcc_send_rec[2] failed: Connection refused
ipcc_send_rec[3] failed: Connection refused
Unable to load access control list: Connection refused
 
halley80 said:
Thank you for your response but:
the /etc/pve directory is empty ....
Click to expand...

Your pmxcfs does not starts, kan you check more of the log for the actual reason:
journalctl -b -u pve-cluster

Or try to start it manually: pmxcfs
 
Hi,

Thank you for your response.
I reinstalled node 6 (cleaner).
I backed up my VM(s), turn off the node and run the command => pvecm delnode.

Then I reinstalled it with the same name/ip.
Is it a problem when I'm going to reintegrate it in the cluster ?

Thank you for your answers,
Yours sincerely,
 
but just the network/interfaces for the Vlan assignment 'Tag VLAN' => Ex: enp94s0f0np0.10
Click to expand...
interfaces name can't be more than 14 characters lenght.

if you need a vlan tag with long interface name:

you can use "vlan10" interface name with vlan-raw-device

Code: 
auto vlan10
iface vlan10 inet static
        address ...
        vlan-raw-device enp94s0f0np0

or if you use ifupdown2, you can also use random name

Code: 
auto mgmt
iface mgmt inet static
        address ..
        vlan-id 10
        vlan-raw-device enp94s0f0np0

(you can manage with gui in last proxmox version)
 
Hello

Thank you for your response.
I understand well for the name of the network legs but I am in a hurry to get back into production.

The network paw names (with Tag Vlan) work on another node.
Do you think I can reintegrate this node without any problem ?

Thank you,
Yours sincerely,
 
Here is the network interfaces.

What do you think about it?
What are the modifications to be made ?

Thanking you,
Sincerely
 

Attachments

  • interfaces.txt
    1.1 KB · Views: 14
try

This should fix the management
Code: 
auto vlan10
iface vlan10 inet manual
        vlan-raw-device enp94s0f0np0

auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp94s0f0np0
    bridge-stp off
    bridge-fd 0

auto vmbr0v10
iface vmbr0v10 inet static
    address  192.168.38.108
    netmask  255.255.255.0
    gateway  192.168.38.1
    bridge_ports vlan10
    bridge_stp off
   bridge-fd 0



Also related, you'll have problems, if you use vms defined on vmbr0 with tag=X, as proxmox also try to create vmbr0vX bridges with enp94s0f0np0.X)

A way to get both full working is to use vlan-aware bridge


Code: 
auto vmbr0
iface vmbr0 inet manual
    bridge_ports enp94s0f0np0
    bridge_stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr0.10
iface vmbr0.10 inet static
    address  192.168.38.108
    netmask  255.255.255.0
    gateway  192.168.38.1
 
Thank you very much.
I will apply this.

However, the cluster communicates on 46.0/24.

Also in the file '/etc/pve/priv/known_hosts' ipmpve6' still appears.
Can this be a problem when adding to the cluster ?
 
Hi,

After a 'fresh_install' of the node.
I applied the network configuration advised/given by 'spirit'.

Many thanks to him.
Best regards,
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back