Glusterfs without NFS

[mcmyst]

Hi,

I am setting up proxmox 3.4 to work with a glusterfs volume.
The volume is UP and running:

gluster volume info
Volume Name: testvol
Type: Replicate
Volume ID: af6a23f2-95fa-4cd1-9357-9529df85cd16
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: nas1:/data/brick1/testvol
Brick2: nas2:/data/brick1/testvol
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on

When I try to add the volume via GUI the volume list is not showing anything.
When I look in the console of the browser, I can see:

[HTTP/1.1 500 command '/sbin/showmount --no-headers --exports 172.16.3.17' failed: exit code 1 70 ms]

This command is looking for a NFS share, I was hoping proxmox would use glusterfs-client (the glusterfs client is better in handling failover node).

How can I configure proxmox to not use NFS when I setup glusterfs volume ?

Thank you

 

[wolfgang]

Hi,

the problem is you have a nfs what block all storage operation.

So you have to take care the nfs will work proper or disable it.

 

[dcsapak]

This command is looking for a NFS share, I was hoping proxmox would use glusterfs-client (the glusterfs client is better in handling failover node).

How can I configure proxmox to not use NFS when I setup glusterfs volume ?

we just use the nfs portion of gluster to show the available volumes, we use the (qemu-builtin) gfapi when using a disk in a virtual machine
if you know the name of your volume you can simply enter it and it should work, even with gluster nfs disabled

 

[mcmyst]

Ok so I have put the name of the volume but it is not working.
Here is the storage.cfg file:

glusterfs: Shared
volume testvol
path /mnt/pve/Shared
content images,iso,vztmpl,rootdir,backup
server2 172.16.3.19
server 172.16.3.17
maxfiles 1

On the GUI when I click on the Shared storage, I can see this error :

mount error: exit code 1 (500)

The glusterfs servers doesn't do any firewalling yet and are on the same network as proxmox.

I don't see any client connected to volume on the gluster servers:

gluster volume status testvol clients
Client connections for volume testvol
----------------------------------------------
Brick : nas1:/data/brick1/testvol
Clients connected : 2
Hostname                                               BytesRead    BytesWritten
--------                                               ---------    ------------
172.16.3.17:49149                                        2582144         2252816
172.16.3.19:49146                                           1676            1052
----------------------------------------------
Brick : nas2:/data/brick1/testvol
Clients connected : 2
Hostname                                               BytesRead    BytesWritten
--------                                               ---------    ------------
172.16.3.17:49146                                           2564            1496
172.16.3.19:49149                                        2581256         2252372
----------------------------------------------

I can mount the volume from another linux client:
mount -t glusterfs 172.16.3.17:/testvol /mnt

 

[dcsapak]

mount -t glusterfs 172.16.3.17:/testvol /mnt

does this also work on the pve host? (please use not /mnt there or all other storages gets overmounted)

we do not have a leading slash in our command, so it would look like this:

mount -t glusterfs 172.16.3.17:testvol /mnt/pve/Shared

but it should work regardless

 

[mcmyst]

Ok so here is the result of the commande:

mount -t glusterfs 172.16.3.17:testvol /mnt/pve/Shared
Mount failed. Please check the log file for more details.

And I found this log file /var/log/glusterfs/mnt-pve-Shared.log:

[2017-03-01 08:26:19.735666] E [name.c:249:af_inet_client_get_remote_sockaddr] 0-testvol-client-0: DNS resolution failed on host nas.mydomain.fr
[2017-03-01 08:26:19.736612] E [common-utils.c:222:gf_resolve_ip6] 0-resolver: getaddrinfo failed (Name or service not known)
[2017-03-01 08:26:19.736633] E [name.c:249:af_inet_client_get_remote_sockaddr] 0-testvol-client-1: DNS resolution failed on host nas2.mydomain.fr

So I put the hostname in my /etc/hosts and on the DNS server and it is working !

Thank you