firewall cluster nodes problem

[marcO0s]

I got a cluster with 5 nodes. Each of them connected via a GRE to allow multicast (because my local network not allow multicast).

Everything work fine and i want to activate firewall on cluster. My problem is that when i start firewall on a node it become red and not accessible by other.

My GRE tunnel seems to be up but no ping , no connection is possible on it.

i see that rules in iptables and i think that's the problem:
Chain PVEFW-DropBroadcast (2 references)
target prot opt source destination
DROP all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST
DROP all -- anywhere anywhere ADDRTYPE match dst-type MULTICAST
DROP all -- anywhere anywhere ADDRTYPE match dst-type ANYCAST
DROP all -- anywhere base-address.mcast.net/4
all -- anywhere anywhere /* PVESIG:NyjHNAtFbkH7WGLamPpdVnxHy4w */

Could you please help me understanding what i should do to make it working ?

Marc.

 

[manu]

Try to add on your datacenter a rules to allow the corosync traffic
you need to allow udp traffic on port 5404 and 5405

 

[marcO0s]

Firstly thanks for your reply,

i've already open udp port 5404 and 5405 in "IN rules" in my datacenter.
Have a look at the picture included for my datacenter configuration if you could find any mistake.

Do you have any other idea ?

 

[manu]

What is the output of pvecm status on your system ?

 

[marcO0s]

What is the output of pvecm status on your system ?

Quorum information
------------------
Date: Mon Oct 24 09:44:34 2016
Quorum provider: corosync_votequorum
Nodes: 5
Node ID: 0x00000001
Ring ID: 4/3660
Quorate: Yes

Votequorum information
----------------------
Expected votes: 5
Highest expected: 5
Total votes: 5
Quorum: 3
Flags: Quorate

Membership information
----------------------
Nodeid Votes Name
0x00000004 1 192.168.0.1
0x00000003 1 192.168.0.2
0x00000001 1 192.168.0.3 (local)
0x00000002 1 192.168.0.4
0x00000005 1 192.168.0.5

 

[manu]

Everything is fine here, so what is the output of

service pve-cluster status

on your nodes ?

 

[marcO0s]

i found the start of the problem.
My problem is that GRE is not accepted by rules and my multicast for cluster is on the GRE

If i run pve-firewall simulate

i got this error :
unable to parse rule: -p gre -j RETURN at /usr/share/perl5/PVE/FirewallSimulator.pm line 245.

I think there is a problem in firewall with protocol gre
If i choose macro GRE instead of protocol gre i got another error :
unable to parse rule: -p 47 -j RETURN at /usr/share/perl5/PVE/FirewallSimulator.pm line 245.

If i create this 2 rules in iptables before starting firewall everything is ok :
iptables -D INPUT -p gre -j ACCEPT
iptables -D OUTPUT -p gre -j ACCEPT

Can you tell me if this is a bug in proxmox 4 ? Do you have any idea on how to solve it ?

 

[marcO0s]

iptables -A INPUT -p gre -j ACCEPT
iptables -A OUTPUT -p gre -j ACCEPT

not -D

 

[manu]

Can you submit a bug report in bugzilla.proxmox.com about this ?

Even if we solve the firewall problem, you should know that the latency of the GRE tunnel might causes problem for corosync.
What is the current latency between your hosts when you ping them over the GRE tunnel ?
Corosync requires very short latency in the 2.3 ms range when pinging a host. What is the output of a ping command between your hosts ?

 

[marcO0s]

we got a nice network
64 bytes from 192.168.0.3: icmp_seq=1 ttl=64 time=0.100 ms

 

[marcO0s]

ok i will open a bug report

 

[marcO0s]

is there a way to write iptables rules before pve-firewall start ? in which files would it be possible ?