HA / Fence Probleme

K

kev1904

Well-Known Member
Feb 11, 2019
61
4
48
32
Hallo Zusammen,

wir haben einige Probleme mit HA bzw Corosync ?

Seit einiger Zeit ist ein Node aus dem Cluster welches aus 8 Servern besteht offline wegen Netzwerk umbauten. Nehmen wir den Server wieder online Kann er 2 Server aus dem Cluster nicht erreichen wegen Netzwerk Problemen. Alle anderen 5 Server können erreicht werden. Müsste Corosync dann nicht die bisher funktionierenden Systeme "In ruhe" lassen und bemängeln das mit dem ersten Server welcher Netzwerk Probleme hat etwas nicht stimmt und diesen Fencen ?
Wir sind da grade etwas aufgeschmissen.

Zur Vereinfachung der Situation

prox1 / OFFLINE Kann prox3 und 6 nicht erreichen
prox2
prox3 Kann von prox1 nicht erreicht werden und wird gefenced wenn prox1 online geht
prox4
prox5
prox6 Kann von prox1 nicht erreicht werden und wird gefenced wenn prox1 online geht
prox7
prox8
 
Wie sind die IPs von den jeweiligen Nodes? Gibt es hier eventuell Konflikte?
Bitte poste die Corosync Config (/etc/pve/corosync.conf).
 
IP Konflikte gibt es nicht.

to_syslog: yes
}

nodelist {
node {
name: prox01
nodeid: 1
quorum_votes: 1
ring0_addr: 192.168.75.1
}
node {
name: prox02
nodeid: 2
quorum_votes: 1
ring0_addr: 192.168.75.2
}
node {
name: prox03
nodeid: 3
quorum_votes: 1
ring0_addr: 192.168.75.3
}
node {
name: prox04
nodeid: 4
quorum_votes: 1
ring0_addr: 192.168.75.4
}
node {
name: prox06
nodeid: 8
quorum_votes: 1
ring0_addr: 192.168.75.6
}
node {
name: proxstore11
nodeid: 9
quorum_votes: 1
ring0_addr: 192.168.75.111
}
node {
name: proxstore12
nodeid: 10
quorum_votes: 1
ring0_addr: 192.168.75.112
}
node {
name: proxstore13
nodeid: 11
quorum_votes: 1
ring0_addr: 192.168.75.113
}
}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: Galaxis
config_version: 21
interface {
bindnetaddr: 192.168.75.1
ringnumber: 0
}
ip_version: ipv4
secauth: on
version: 2
}
 
Bitte den Output von ip a für Nodes prox1, prox3 und prox6 posten. Sollten Public IPs dabei sein, können diese entfernt werden.
 
Prox1
root@prox01:/etc/network# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 5c:f3:fc:e4:cb:84 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 5c:f3:fc:e4:cb:86 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 5c:f3:fc:6b:06:34 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 5c:f3:fc:6b:06:36 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:10:18:a6:06:30 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:10:18:a6:06:32 brd ff:ff:ff:ff:ff:ff
8: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:10:18:a6:06:34 brd ff:ff:ff:ff:ff:ff
9: eth7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:10:18:a6:06:36 brd ff:ff:ff:ff:ff:ff
10: eth8: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
11: eth9: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
12: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 5e:f3:fc:dc:cb:87 brd ff:ff:ff:ff:ff:ff
13: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
14: vlan75@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.1/24 brd 192.168.75.255 scope global vlan75
valid_lft forever preferred_lft forever
inet6 fe80::ee0d:9aff:fe1d:fb60/64 scope link
valid_lft forever preferred_lft forever
15: vlan76@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
inet 192.168.76.1/24 brd 192.168.76.255 scope global vlan76
valid_lft forever preferred_lft forever
inet6 fe80::ee0d:9aff:fe1d:fb60/64 scope link
valid_lft forever preferred_lft forever
16: vlan302@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
inet 172.19.76.1/16 brd 172.19.255.255 scope global vlan302
valid_lft forever preferred_lft forever
inet6 fe80::ee0d:9aff:fe1d:fb60/64 scope link
valid_lft forever preferred_lft forever
17: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ec:0d:9a:1d:fb:60 brd ff:ff:ff:ff:ff:ff
inet6 fe80::ee0d:9aff:fe1d:fb60/64 scope link
valid_lft forever preferred_lft forever
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 00:10:18:be:1e:88 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond1 state UP group default qlen 1000
link/ether 00:10:18:be:1e:8a brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond2 state UP group default qlen 1000
link/ether 5c:f3:fc:6b:24:dc brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond2 state UP group default qlen 1000
link/ether 5c:f3:fc:6b:24:dc brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 00:10:18:be:1e:88 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond1 state UP group default qlen 1000
link/ether 00:10:18:be:1e:8a brd ff:ff:ff:ff:ff:ff
8: eth6: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond2 state UP group default qlen 1000
link/ether 5c:f3:fc:6b:24:dc brd ff:ff:ff:ff:ff:ff
9: eth7: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond2 state UP group default qlen 1000
link/ether 5c:f3:fc:6b:24:dc brd ff:ff:ff:ff:ff:ff
10: eth8: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ec:0d:9a:1d:e7:c0 brd ff:ff:ff:ff:ff:ff
11: eth9: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ec:0d:9a:1d:e7:c1 brd ff:ff:ff:ff:ff:ff
12: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 5e:f3:fc:c0:96:7f brd ff:ff:ff:ff:ff:ff
13: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:1e:88 brd ff:ff:ff:ff:ff:ff
inet6 fe80::210:18ff:febe:1e88/64 scope link
valid_lft forever preferred_lft forever
14: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:1e:8a brd ff:ff:ff:ff:ff:ff
inet6 fe80::210:18ff:febe:1e8a/64 scope link
valid_lft forever preferred_lft forever
15: vlan75@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:1e:88 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.3/24 brd 192.168.75.255 scope global vlan75
valid_lft forever preferred_lft forever
inet6 fe80::210:18ff:febe:1e88/64 scope link
valid_lft forever preferred_lft forever
16: vlan76@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:1e:8a brd ff:ff:ff:ff:ff:ff
inet 192.168.76.3/24 brd 192.168.76.255 scope global vlan76
valid_lft forever preferred_lft forever
inet6 fe80::210:18ff:febe:1e8a/64 scope link
valid_lft forever preferred_lft forever
17: vlan302@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:1e:8a brd ff:ff:ff:ff:ff:ff
inet 172.19.76.3/16 brd 172.19.255.255 scope global vlan302
valid_lft forever preferred_lft forever
inet6 fe80::210:18ff:febe:1e8a/64 scope link
valid_lft forever preferred_lft forever
ip a
10: eth8: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ec:0d:9a:1d:d6:60 brd ff:ff:ff:ff:ff:ff
11: eth9: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ec:0d:9a:1d:d6:61 brd ff:ff:ff:ff:ff:ff
12: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:20:c8 brd ff:ff:ff:ff:ff:ff
inet 192.168.75.6/24 brd 192.168.75.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::210:18ff:febe:20c8/64 scope link
valid_lft forever preferred_lft forever
13: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:20:ca brd ff:ff:ff:ff:ff:ff
inet6 fe80::210:18ff:febe:20ca/64 scope link
valid_lft forever preferred_lft forever
14: vlan76@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:20:ca brd ff:ff:ff:ff:ff:ff
inet 192.168.76.6/24 brd 192.168.76.255 scope global vlan76
valid_lft forever preferred_lft forever
inet6 fe80::210:18ff:febe:20ca/64 scope link
valid_lft forever preferred_lft forever
15: vlan302@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:10:18:be:20:ca brd ff:ff:ff:ff:ff:ff
inet 172.19.76.6/16 brd 172.19.255.255 scope global vlan302
valid_lft forever preferred_lft forever
inet6 fe80::210:18ff:febe:20ca/64 scope link
valid_lft forever preferred_lft forever
16: bond2: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
link/ether 5c:f3:fc:e9:11:62 brd ff:ff:ff:ff:ff:ff
17: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 5c:f3:fc:e9:11:62 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5ef3:fcff:fee9:1162/64 scope link
valid_lft forever preferred_lft forever
 
wieso prox1 den prox3 und 6 nicht sieht prüfen wir grade, evtl ein bug im Treiber der neu verbauten 2x10G nic. Uns geht es eher darum, wieso ha den prox 3 und 6 fenced obwohl alle anderen nodes diese erreichen können, und lediglich mit prox01 etwas nicht stimmt.
 
Wie sieht es denn bei den anderen Nodes mit VLAN aus? Hier scheint ja prox6 nicht im selben VLAN zu sein wie die anderen beiden Nodes.
 
Bei manchen ist das vlan75 tagged bei manchen ist es untagged (native) aktuell sind alle nodes mit mehreren Kupfer LACP channel verbunden. wir wollen das Kupfer gegen glas ersetzen, deswegen der prox1 als test. Die Konnektivität zwischen allen anderen ist ja gegeben, bei prox1 war das vorher mit Kupfer auch alles OK erst seit dem wir nun glas eingebaut haben buggt es rum, aber wir verstehen nicht wieso node 3 und 6 fanced werden obwohl prox01 offensichtlich der Übeltäter ist
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back