NFS from OmniOS permission errors

[carlosmp]

Hi,

I found a few articles specific to OmniOS, specifically this one - http://forum.proxmox.com/threads/15446-OpenVZ-Container-in-Napp-IT-OmniOS-NFS, but I am still unable to restore to the NFS storage devices.

Quick checklist:
- options vers=3,tcp in /etc/pve/storage.conf for my nfs volumes

- sharenfs properties for nfs volumes (have 2 one for private/images and the other for backups/templates

root@rundc1-zfs01:/pool1# zfs get sharenfs /pool1/pvestorage
NAME              PROPERTY  VALUE                    SOURCE
pool1/pvestorage  sharenfs  rw,root=@172.30.30.0/24  local

root@rundc1-zfs01:/pool1# zfs get sharenfs /pool1/pvebackup 
NAME             PROPERTY  VALUE                    SOURCE
pool1/pvebackup  sharenfs  rw,root=@172.30.30.0/24  local

- ls -V for pool directories

root@rundc1-zfs01:/pool1# /usr/bin/ls -V /pool1/pvestorage/
total 132
drwxrwxrwx+  2 root     root           2 Oct 28 20:01 dump
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWc--s:fd-----:allow
drwxrwxrwx+  2 root     root           2 Oct 28 20:01 images
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWc--s:fd-----:allow
drwxrwxrwx+  2 root     root           2 Oct 30 00:18 private
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWcCos:fd-----:allow
drwxrwxrwx+  4 root     root           4 Oct 28 20:01 template
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWc--s:fd-----:allow
root@rundc1-zfs01:/pool1# /usr/bin/ls -V /pool1/pvebackup  
total 132
drwxrwxrwx+  3 root     root         260 Oct 29 22:48 dump
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWc--s:fd-----:allow
drwxrwxrwx+  2 root     root           2 Oct 27 08:38 images
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWc--s:fd-----:allow
drwxrwxrwx+  2 root     root           2 Oct 30 00:15 private
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWcCos:fd-----:allow
drwxrwxrwx+  4 root     root           4 Oct 27 08:38 template
              user:root:rwxpdDaARWcCos:fd-----:allow
              everyone@:rwxpdDaARWc--s:fd-----:allow

- From proxmox, if I mkdir in one of the mount points, the directory is created correctly, the only difference is the Inherit property (I)?

drwxrwxrwx+  2 root     root           2 Oct 30 00:24 test
              user:root:rwxpdDaARWcCos:fd----I:allow
              everyone@:rwxpdDaARWc--s:fd----I:allow

- hosts file from proxmox node(s)

root@pve01:~# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
172.30.30.90 pve01.runcentral.com pve01 pvelocalhost
172.30.30.91 rundc1-pve02.runcentral.com rundc1-pve02

- hosts from omnios

::1             localhost
127.0.0.1       localhost loghost
127.0.0.1       rundc1-zfs01
172.31.31.14     pFFFF.ibp0
172.30.30.33     rundc1-zfs01
127.0.0.1       rundc1-zfs01

- I've rebooted the OmniOS box after all teh changes.

- I've tried to unmount/remount on proxmox by unchecking the Enable, running umount from the CLI on proxmox, and then re-enabling in Proxmox. This seems to remount the point.

When trying to restore (to migrate storage) or create new container, I'm getting this in the window: (from restore)

extracting archive '/mnt/pve/zfs01-pvebackup/dump/vzdump-openvz-109-2013_10_29-21_56_19.tar.lzo'
tar: ./opt: Cannot change mode to rwxr-xr-x: Operation not permitted
tar: ./usr/sbin/iptables-apply: Cannot open: Operation not permitted
tar: ./usr/sbin/setvesablank: Cannot open: Operation not permitted
...
tar: ./aquota.user: Cannot open: Operation not permitted
tar: ./run/mount/utab: Cannot open: Operation not permitted
tar: ./run/mount: Cannot change mode to rwxr-xr-x: Operation not permitted
tar: ./run/.run-transition: Cannot open: Operation not permitted
tar: ./run: Cannot change mode to rwxr-xr-x: Operation not permitted
Total bytes read: 2325770240 (2.2GiB, 54MiB/s)

Not sure what else to try. It seems the only thing I can't get to explicitly work is the no_root_squash, but that seems to be an invalid option in OmniOS...

Thanks in advance,

Carlos

 

[mir]

Read: http://forum.proxmox.com/threads/15793-CT-creation-on-NFS-Share?p=81530#post81530

 

[carlosmp]

That was it...

zfs set aclmode=passthrough
zfs set aclinherit=passthrough-x

Thanks for the pointer. I had been searching around, and most of the stuff started to look the same...

 

[Raymond Burns]

How do I create a WIKI for this? I went days around with this

 

[mir]

1) Create an account on the wiki and log in.
2) In the search field to the left write the title of your page and the search. Result is a new page ready to edit.

 

[carlosmp]

Been doing some testing and roll outs, and having a few issues with the NFS side. I've got 3 different NFS systems. OmniOS, OpenFiler, and FreeNAS. The FreeNAS system appears to be working correctly. The other two systems are not. The symptoms we're seeing are:
- on container creation, resolv.conf and hosts are not being created correctly from the GUI.
- getting weird errors when running yum, claiming not enough space, and needing 17GB to install webmin.

On FreeNAS - the settings there are:
- ownder user/group = root / wheel
- mode - owner:rwx, group:rx, other:rx
- ACL - unix
- NFS Share permissions are auth networks 172.30.30.0/24, maproot user/group are root/wheel.

On OmniOS, the settings are as above. I'm trying to get the settings from OpenFiler, as it's not a system i have direct control on, but the last option we enabled there was the no root squash.

My guess would be that perhaps proxmox is creating certain things as a different user that OmniOS/OpenFiler are more sensitive to?

Carlos.

 

[Raymond Burns]

I had the same exact issues with webmin.
Basically, I had to install the dependency requesting more space, by itself. EG: if mysql-server requires 17gb, then install mysql-server first. Then try to install webmin and it's dependencies. I had to install roughly 3 or 4 before webmin would install without a problem.

 

[carlosmp]

That works for a while, until I get kernel-devel, which is 8.2M and says it needs another 3578MB on the / filesystem. Resorted to increasing the partition to 16GB to let that last package work, then we can shrink it . Definitely something strange between proxmox and the NFS system. openfiler and OmniOS must be missing something...