Bug Glusterstorage

[badji]

* I have 3 nodes proxmox.
* Glusterfs-server works in 3 nodes without Iptables
* with iptables Glusterfs-server works but just in 2 nodes. with commande i don't see problem with gluster and brick.
Some help? Thank's.

 

[dcsapak]

what are your firewall rules ?

 

[badji]

My firewall rules from glusterfs :

# On autorise les connexions GLUSTER comme SAN
iptables -t filter -A INPUT -p tcp --dport 111 -j ACCEPT
iptables -t filter -A OUTPUT -p tcp --dport 111 -j ACCEPT
iptables -t filter -A INPUT -p udp --dport 111 -j ACCEPT
iptables -t filter -A OUTPUT -p udp --dport 111 -j ACCEPT
iptables -t filter -A INPUT -p tcp -m multiport --dport 24007:24020 -j ACCEPT
iptables -t filter -A OUTPUT -p tcp -m multiport --dport 24007:24020 -j ACCEPT
iptables -t filter -A INPUT -p tcp --dport 49152 -j ACCEPT
iptables -t filter -A OUTPUT -p tcp --dport 49152 -j ACCEPT
echo " -Ports Gluster 111 24007 24008 24009 49152 Ok "

Merci.

 

[dcsapak]

you are missing the nfs port
2049 For glusterFS's NFS exports (nfsd process).

 

[badji]

Thank's Dominique,
Sorry i don't miss 2049, i use it in rule for NFS directly :

# On autorise la connexion nfs mount
iptables -t filter -A INPUT -p tcp --dport 111 -j ACCEPT
iptables -t filter -A OUTPUT -p tcp --dport 111 -j ACCEPT
iptables -t filter -A INPUT -p udp --dport 111 -j ACCEPT
iptables -t filter -A OUTPUT -p udp --dport 111 -j ACCEPT
iptables -t filter -A INPUT -p tcp --dport 2049 -j ACCEPT
iptables -t filter -A OUTPUT -p tcp --dport 2049 -j ACCEPT
iptables -t filter -A INPUT -p udp --dport 2049 -j ACCEPT
iptables -t filter -A OUTPUT -p udp --dport 2049 -j ACCEPT
iptables -t filter -A INPUT -p tcp --dport 20048 -j ACCEPT
iptables -t filter -A OUTPUT -p tcp --dport 20048 -j ACCEPT
iptables -t filter -A INPUT -p udp --dport 20048 -j ACCEPT
iptables -t filter -A OUTPUT -p udp --dport 20048 -j ACCEPT
echo " -Port 111 2049 20048 OK"

The problem : Glustefs work's just for 2 nodes ( With Iptables), when i try with 3 nodes, if i add in /etc/pve/storage
server3 192.168.1.63 i have this message :

root@pve3:/etc/init.d# pvesm status
file /etc/pve/storage.cfg line 23 (section 'glusterfs') - unable to parse value of 'server3': unexpected property 'server3'
storage 'glusterfs' is not online
Name Type Status Total Used Available %
ceph-storage_ct rbd active 1411383296 16697344 1394606080 1.18%
ceph-storage_vm rbd active 1411383296 16697344 1394606080 1.18%
glusterfs glusterfs inactive 0 0 0 0.00%
local dir active 216589696 17500672 199089024 8.08%
local-zfs zfspool active 199089140 96 199089044 0.00%

ps: With the same rules firewall, gluster run just from 2 nodes, never with the third.

Merci.

 

[dcsapak]

first, there is no 'server3' option for glusterfs (it is only possible to specify 2 servers atm)

to check if a server is online we try to connect to port 7, so it seems this has to be accepted in the firewall as well

 

[badji]

Merci,
I added a server NFS with Centos and it works on the 3 servers. But Glusterfs only works on the first two !!!
Maybe it takes another port by brick?