GlusterFS - ISO and Redundant Storage Issues

[MGB]

Hi,

I'm testing a 4 Node PVE cluster using gluster as shared storage. (Version 3.1-14 - non subscription)

Two things I noticed which seem like bugs.

1. Can not upload ISO to the gluster store. (Error 500: cant upload to storage type 'glusterfs').
2. I have three mirrored gluster nodes. I set proxmox glusterfs plugin to mount the gluster volume at gluster Node1 volume gluster0. When I power off gluster node 1 the other 3 proxmox nodes can no longer run VMs or containers on the shared gluster0 volume. However if I ssh to any of the powered on proxmox nodes and check /mnt/pve/gluster0 all files in the gluster volume are accessible. So although the mount point seems to have correctly failed over to one of the two remaining gluster nodes, proxmox glusterfs plugin has not followed suite. Also if I try and view gluster0 through the proxmox web interface I get a volume offline error. After powering gluster node 1 back on PVE finds gluster0 and all is well again. However, this issue seems to prevent gluster from being used as a shared HA store.

 

[spirit]

Hi,

1. seem to be a bug in pve-manager, can you fill a bugzilla ?

2. qemu access directly to gluster without using the filesystem mountpoint. (filesystem is only used to manage disk creation/delete/snapshot/....)
How does the failover of a mountpoint works ? (I think you can mount only 1 ip address by mountpoint ?)
As workaround, maybe install gluster on each proxmox node, and use localhost as mount point could work ?

 

[MGB]

Hi Spirit,

1. Thanks, have created bug ID 465.
2. That's a good idea to use localhost. In this test environment 3 of the 4 nodes are running gluster server, so I think your idea should work for these 3 nodes (and I plan to test later today). If it does work it would mean gluster server and replicated volumes would need to run on all nodes in the cluster.

 

[spirit]

1. If it does work it would mean gluster server and replicated volumes would need to run on all nodes in the cluster.

I really don't known if we can install a gluster daemon on 1 node, without using it for replication. (something like sheepdog gateway mode).

 

[MGB]

1. 
   
   I really don't known if we can install a gluster daemon on 1 node, without using it for replication. (something like sheepdog gateway mode).

1. 
   
   Local host worked for the three nodes, as expected the fourth could not find the gluster server. I'll install the gluster server on the 4th node and add it to the gluster cluster without actually setting up a replicated volume. The other thought I had was setting up some kind of fail over IP between two of the nodes running gluster server.

 

[arifkhumaidi]

helloo..
have this bug really been solved??
"Can not upload ISO to the gluster store. (Error 500: cant upload to storage type 'glusterfs')."