Gluster packages form gluster.org

[dev-null]

Dear all,


there are several posts about problems with gluster and gluster mounts, especially when rebooting the node that was inserted as glusterfs server.
We are investigating this issue and first of all we wonder if there would be any problems updating gluster with packages from gluster.org


Environment:


PVE 4.0
glusterfs-client 3.5.2-2+deb8u1 amd64 clustered file-system (client package)
glusterfs-common 3.5.2-2+deb8u1 amd64 GlusterFS common libraries and translator modules


Gluster Server


glusterfs-client 3.5.2-2+deb8u1 amd64 clustered file-system (client package)
glusterfs-common 3.5.2-2+deb8u1 amd64 GlusterFS common libraries and translator modules
glusterfs-server 3.5.2-2+deb8u1 amd64 clustered file-system (server package)


The gluster is running on 3 identical servers with replica=3, previously we were running CentOS on the same servers with gluster 3.7 and all the gluster mounts did not have a problem at all, now with version 3.5.2 and promox there are frequent problems.


We are thinking of 2 different scenarios


1. just update the gluster and keep using the PVE GUI for the mount


2. do a manual mount and use "directory" in the PVE Gui Does anyone have input or remarks on this ?


Regards Soeren

 

[aderumier]

>>1. just update the gluster and keep using the PVE GUI for the mount

If you are happy with gluster 3.7,yes,

you can simply add 3.7 debian repository to proxmox

http://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/jessie/

They are no specific proxmox patches on current packages. (they are the packages from debian repository)


>>2. do a manual mount and use "directory" in the PVE Gui Does anyone have input or remarks on this ?

Do do that, it's using fuse, it's slow.
proxmox gluster plugin use gluster api directly from qemu.



What kind of problem do you have with 3.5 ? (I ask that because this is the stable branch)

 

[dev-null]

Dear aderumier,

first, thanks for the answer, our problem with the current setup is, that we have VMS (especially with larger volumes, >400GB) and especially Windows frequently have problems writing to the disk making that disk readonly and subsequently the complete VM stops working.

Regarding "2." i assume that you wanted to type "Don't do that" - i agree that fuse tends to be slower, however in the ovirt setup, the mount is done through fuse as well even though it uses qemu for the virtual machines - and fomr our perspective that was much more stable.

One of the biggest problems for regarding this is, that we can not find evidence in logfiles indicating what really happens, we are not even sure if it is the gluster that causes the problem, the only irregular thing we can find is this

got unexpected error - can't open '/etc/pve/nodes/mc-dc3ham-compute-04-live/lrm_status' - No such file or directory

but at totally different times.

Any idea how to better debug this ?

Cheers
Soeren

 

[t.lamprecht]

got unexpected error - can't open '/etc/pve/nodes/mc-dc3ham-compute-04-live/lrm_status' - No such file or directory

but at totally different times.

As a Note: this normally does not correlate with a storage problem, it's from the HA manager when he tries to read the local resource manager status (thus lrm) when the lrm was not started at least once (with a resource configured). This can be ignored normally but I'll take a look at this.

As aderumier said try the newest packages from the glusterFS jessie mirror, using raw disk format with windows also may help, if not already selected.

 

[dev-null]

Thanks Thomas, i did not expect this to be one of the problems.

Regarding the original problem, we decided to go for

- manual mount
- glusterfs 3.7

We will keep you posted

cheers
soeren

 

[dev-null]

Hi,

i can already say that we are having contiuous problems with the "normal" setup, whenever we put a little bit of load on the system, we experience outages in VMS, specifically that the image files are readonly for the VM.

We have several icinga checks running for network connectivity - there is no problem, but a bit of load and the load on the hypervisor host as well as on the virtual machine shows massive spikes and the it flatlines on the VM since the volume is in read only.

We do not know where to look anymore

Cheers
Soeren