NFS Datastore: EINVAL: Invalid argument

[MAgno Santos]

Hello,

I use a FreeNAS external Box which currently host PromoxVE vzdump backup + Windows VM Guest Backups (with Veeam free agent or other tecnologies).
I can mount the NFS share on the PBS server and rw from it for instance at /mnt/nas.

When i try to create the datastore at PBS i have the error "EINVAL: Invalid argument"

 

[jayg30]

I'm seeing the same thing.
Is this an NFS permission issue?
Share is NFSv4. Same shares work fine added to PVE for vzdump.

 

[alebeta]

Hi,

am using NFS shares as datastore, if you could share your export config on the NFS server, maybe I could help.

All the best

 

[jayg30]

Hi,

am using NFS shares as datastore, if you could share your export config on the NFS server, maybe I could help.

All the best

I suspect you're probably using linux as your NFS server.
FreeNAS is a bit different when setting up NFS shares.

Export file that's generated by FreeNAS GUI;

/mnt/store/pve/pbs -quiet -maproot="root":"wheel" -network X.X.X.X/24

Output of mount on PBS server;

freenas:/mnt/store/pve/pbs on /mnt/pbs-store-01 type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=X.X.X.X,local_lock=none,addr=X.X.X.X)

I just also noticed if I provide a non-existing folder path when trying to create the datastore, that folder is created and then the error complains with some more info;

root@pbs:~# proxmox-backup-manager datastore create pbs-store-01 /mnt/pbs-store-01/pve
Error: unable to create chunk store 'pbs-store-01' at "/mnt/pbs-store-01/pve" - EINVAL: Invalid argument
root@pbs:~# ls -alh /mnt/pbs-store-01/
total 5.0K
drwxrwxrwx 3 root 4294967294    3 Jul 25 14:03 .
drwxr-xr-x 4 root root       4.0K Jul 25 13:11 ..
drwxr-xr-x 2 root 4294967294    2 Jul 25 14:03 pve

As noted, this setup works fine for PVE. I use it to provide all type of storage to PVE. Images, templates, containers, VMs, and vzdump. From PBS I can even write data to the mounted NFS share without problem. I'm guessing the datastore gets setup using the "backup" user (id=34) on PBS and perhaps that's part of my problem here. There isn't a lot I can alter on FreeNAS for this though.

 

[jayg30]

So it was the permissions issue. I "made" it work by creating a user and group on Freenas called backup for ID=34 and assigning it permission to the ZFS dataset that I'm presenting over NFS. now the permissions show backup:backup on the NFS share in PBS.

root@pbs:~# ls -alh /mnt/
total 13K
drwxr-xr-x  4 root   root   4.0K Jul 25 13:11 .
drwxr-xr-x 18 root   root   4.0K Jul 24 23:24 ..
drwxr-xr-x  2 root   root   4.0K Jul 24 23:23 hostrun
drwxrwx---  3 backup backup    4 Jul 25 18:47 pbs-store-01

root@pbs:~# ls -alh /mnt/pbs-store-01/
total 17M
drwxrwx---     3 backup backup    4 Jul 25 18:47 .
drwxr-xr-x     4 root   root   4.0K Jul 25 13:11 ..
drwxrwx--- 32767 backup backup  65K Jul 25 18:47 .chunks
-rw-r--r--     1 backup backup    0 Jul 25 18:47 .lock

I guess for those of you running Linux for your NFS server you are use the squash parameters or something to get it to work. In FreeNAS/BSD you don't have that. Instead you are supposed to use "maproot user/maproot group" or the big guns "mapall user/mapall group". I'm guessing this didn't really work because I had them set to "root" and "wheel" but PBS is trying to run things as the backup user (id=34) instead. Without an equivalent user ID on the NFS server I don't know how else to even make it work. Hopefully someone else has a more elegant solution to all this.

I simply use maproot user/maproot group on PVE set to root/wheel respectively. I guess that works since there because tasks are run as root on PVE?

I could be completely wrong on all this, NFS permissions confuse me sometimes.

What would be nice is a way to set the user:group to something other than the PBS backup:backup. When they add AD/LDAP alongside NFSv4 and krb5 authentication, I imagine I could leverage that since my FreeNAS is also tied to my AD/LDAP.

 

[jayg30]

Found another approach to make this work.
In FreeNAS go to Service > NFS
Enable NFSv3 ownership model for NFSv4
Description for this setting is;

Set when NFSv4 ACL support is needed without requiring the client and the server to sync users and groups.

Set Maproot User=root
Set Maproot Group=wheel

I also set ACL permissions on the ZFS dataset in FreeNAS that I'm using for the backups. I just set them to root/wheel. Don't think it matter though.

Now you can mount the NFS share under PBS and when you go to add it as a datastore the permissions will be altered as noted here for pbs-store-02.

root@pbs:~# ls -alh /mnt/
total 13K
drwxr-xr-x  5 root   root 4.0K Jul 25 20:23 .
drwxr-xr-x 18 root   root 4.0K Jul 24 23:24 ..
drwxr-xr-x  2 root   root 4.0K Jul 24 23:23 hostrun
drwxrwx---  2 root   root    2 Jul 25 20:29 pbs-store-02

root@pbs:~# proxmox-backup-manager datastore create pbs-store-02 /mnt/pbs-store-02
Percentage done: 1
Percentage done: 2
...
Percentage done: 99

root@pbs:~# ls -alh /mnt/
total 13K
drwxr-xr-x  5 root   root   4.0K Jul 25 20:23 .
drwxr-xr-x 18 root   root   4.0K Jul 24 23:24 ..
drwxr-xr-x  2 root   root   4.0K Jul 24 23:23 hostrun
drwxrwx---  3 backup backup    4 Jul 25 20:32 pbs-store-02

 

[MAgno Santos]

That did the trick

 

[Apollon77]

I have basically the same problem I think ... but with a Synology. He creates the directory but then it seems he wants to execute the "chown" to set owner to backup.backup (which is unneeded because rights are drwxrwxrwx and I think this fails ...

I did not found a proper way to setup that in synology that it preservs the owner :-(

 

[oversite]

I have basically the same problem I think ... but with a Synology. He creates the directory but then it seems he wants to execute the "chown" to set owner to backup.backup (which is unneeded because rights are drwxrwxrwx and I think this fails ...

I did not found a proper way to setup that in synology that it preservs the owner :-(

I had to set NFS options in synology to: squash Map root to admin.
I mounted with these options:
nfs4 rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.xxx,local_lock=none,addr=192.168.1.xxx,insecure 0 0

 

[Apollon77]

For me it was in the end orking with "map to "nothing"" ... but then I ran into the next problem with ext4 and too less number of directories

 

[Apollon77]

I used now a volume with BTRFS as file system and it was working

 

[TechBrain64]

Guys,

I was just battling with this very NFS issue with PBS and TrueNAS. I discovered @jayg30's suggestion works. However, several online blog post recommend setting ownership and permissions using chown and chmod. If you do this PBS will continue to through "EINVAL: Invalid argument".

What I did was set TrueNAS permissions as @jayg30 suggest --> remove the mount --> delete mount-point folder --> recreate mount-point folder (ensure default permissions are set) --> re-mount NFS share --> add datastore "proxmox-backup-manager datastore create <datastore-name> /mnt/<mount-point-folder>

Proxmox Devs: when are you guys going to create a better experience for this issue?

 

[dcsapak]

Proxmox Devs: when are you guys going to create a better experience for this issue?

not sure what we can do to improve the situation here. the path needs to be read/writeable by the 'backup' user/group, if the nfs share is not configured for that to work we can't do anything from pbs side

besides that, we don't recommend having the datastore on an nfs share (or any fileshare for that matter) since it will only add latency/reduce bandwidth which in turn will reduce performance significantly...