[SOLVED] ZFS snaphost as normal user: permission denied

Discussion in 'Proxmox VE: Installation and configuration' started by maxprox, Nov 6, 2018.

  1. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    Hello,

    as a non root user I will work with snapshots, the name of the user is "zrep"

    zrep is also the name of the snapshot tool, witch I use
    because zrep have a look at this both sites:
    http://www.bolthole.com/solaris/zrep/zrep.documentation.html

    https://github.com/olevole/zrep

    Here is what I have done:

    Code:
    zfs allow zrep mount,snapshot,destroy,receive,clone,create,send,rename,hold rpool/data/vm-207-disk-1
    
    Is there an option like " zfs allow all"?

    Code:
    PATH=$PATH:/sbin/
    PATH=$PATH:/usr/sbin/
    PATH=$PATH:/usr/local/sbin
    
    But I get the following ERROR:

    Code:
    zrep@fcprox:/root$ /usr/sbin/zrep clear rpool/data/vm-207-disk-1
    WARNING: Removing all zrep configs and snapshots from
    rpool/data/vm-207-disk-1
     (for TAG=zrep)
    Continuing in 10 seconds
    Destroying any zrep-related snapshots from rpool/data/vm-207-disk-1
    Removing zrep-related properties from rpool/data/vm-207-disk-1
    cannot inherit zrep:src-fs for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:master for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:lock-time for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:dest-host for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:src-host for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:lock-pid for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:dest-fs for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:savecount for 'rpool/data/vm-207-disk-1': permission denied
    
    Has anyone an idea, witch zfs allow option I neet?

    regards,
    maxprox
     
    #1 maxprox, Nov 6, 2018
    Last edited: Nov 9, 2018
  2. WhiteStarEOF

    WhiteStarEOF Member

    Joined:
    Mar 6, 2012
    Messages:
    92
    Likes Received:
    9
    I'm not familiar with zrep, but I'm wondering if it was designed with a dedicated user in mind. Your delegation command looks good. Grab a snapshot by running,

    zfs list -t snapshot -r rpool/data/vm-207-disk-1

    Then see if you can destroy the snapshot using your dedicated user.

    zfs destroy rpool/data/vm-207-disk-1@however-zrep-names-snapshots
     
  3. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    Hi,

    yes, that works:

    Code:
    zrep@fcprox:/root$ zfs list -t snapshot -r rpool/data/vm-207-disk-1
    NAME                                                            USED  AVAIL  REFER  MOUNTPOINT
    rpool/data/vm-207-disk-1@autosnap_2018-11-03_13:26:01_monthly  1.81G      -  61.7G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-05_00:00:01_daily       0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_00:00:01_daily       0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_16:00:01_hourly      0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_17:00:01_hourly      0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_18:00:01_hourly      0B      -  61.8G  -
    
    and destroy the 18:00:01 snapshot as user "zrep":

    Code:
    zrep@fcprox:/root$ zfs destroy rpool/data/vm-207-disk-1@autosnap_2018-11-06_18:00:01_hourly 
    The 18:00 Snapshot is going to nirwana:

    Code:
    zrep@fcprox:/root$ zfs list -t snapshot -r rpool/data/vm-207-disk-1
    NAME                                                            USED  AVAIL  REFER  MOUNTPOINT
    rpool/data/vm-207-disk-1@autosnap_2018-11-03_13:26:01_monthly  1.81G      -  61.7G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-05_00:00:01_daily       0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_00:00:01_daily       0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_16:00:01_hourly      0B      -  61.8G  -
    rpool/data/vm-207-disk-1@autosnap_2018-11-06_17:00:01_hourly      0B      -  61.8G  -
    
    But the zrep tool dos not destroy the snapshots, it will change the properties:

    Code:
    Removing zrep-related properties from rpool/data/vm-207-disk-1
    
    But I do not understand exactly what the zrep tool would like to do in the following lines:

    Code:
    cannot inherit zrep:src-fs for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:master for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:lock-time for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:dest-host for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:src-host for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:lock-pid for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:dest-fs for 'rpool/data/vm-207-disk-1': permission denied
    cannot inherit zrep:savecount for 'rpool/data/vm-207-disk-1': permission denied
    
    an witch rights for this are needed ?

    regards,
    maxprox
     
    #3 maxprox, Nov 6, 2018
    Last edited: Nov 6, 2018
  4. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    Hello,

    also the first initialization snapshot does not work with "permission denied" as user "zrep":
    (I clear the needed properties bevor it as root)

    Code:
    zrep@fcprox:/root$ /usr/sbin/zrep -i rpool/data/vm-207-disk-1 192.168.8.100 r5pool/snap_test/win7lex
    WARNING: no proper recv -o detected
    WARNING: extremely old versions of ZFS crash with volume init
    Continuing in 5 seconds....
    Setting properties on rpool/data/vm-207-disk-1
    cannot set property for 'rpool/data/vm-207-disk-1': permission denied
    cannot set property for 'rpool/data/vm-207-disk-1': permission denied
    cannot set property for 'rpool/data/vm-207-disk-1': permission denied
    cannot set property for 'rpool/data/vm-207-disk-1': permission denied
    cannot set property for 'rpool/data/vm-207-disk-1': permission denied
    Warning: zfs recv lacking -o readonly
    Creating readonly destination filesystem as separate step
    bash: zfs: command not found
    Error: Cannot create 192.168.8.100:
    
     
  5. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,204
    Likes Received:
    498
    check the 'zfs' man page - you likely also need to allow your zrep user to set certain (/all?) properties, as indicated by the error messages ('set' and 'inherit' operations fail with 'permission denied'). not sure what is running on your target host and whether the warning about not supporting 'zfs recv -o' is accurate or an artifact of the check failing because of missing permissions, but that might also be worth checking out.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    Hi fabian,

    yes I have done so, and I found a way to show the permissions

    Code:
    root@fcprox:~# zfs allow rpool/data/vm-207-disk-1
    ---- Permissions on rpool/data/vm-207-disk-1 -------------------------
    Local+Descendent permissions:
            user zrep aclinherit,clone,create,destroy,hold,mount,receive,rename,send,snapshot,userprop
    ---- Permissions on rpool/data ---------------------------------------
    Local+Descendent permissions:
            user zrep aclinherit,clone,create,destroy,hold,mount,receive,rename,send,snapshot,userprop
    
    The missed permission for me was the "userprop"
    (it was a Tip from Philip Brown, who developed zrep)
    The error message "permission denied" now is solved.


    But Now
    Code:
    bash: zfs: command not found
    
    where and how I have to set the PATH variables? any Ideas?
    I try it with
    Code:
    PATH=$PATH:/sbin/
    PATH=$PATH:/usr/sbin/
    PATH=$PATH:/usr/local/sbin
    Code:
    zrep@fcprox:/root$ echo $PATH
    /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    
    also in the users .profile but it does not work, maby it is not the right way because of the bash script?
     
    #6 maxprox, Nov 6, 2018
    Last edited: Nov 7, 2018
  7. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,204
    Likes Received:
    498
    did you set the PATH on both sides? looks to me like the error comes from the receiving end..
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    Yes, I think that is the problem.

    But I have set on both side the PATH variable, and on both side I get the following answer:
    Code:
    zrep@fcprox:~$ echo $PATH
    /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    zrep@fcprox:~$ which zfs
    /sbin/zfs
    
    on both side works the ssh key, and on both side I set the permissions,
    on both side the user zrep can work with the command zfs,
    but I get the "command not found" messages
    Code:
    zrep@fcprox:~$ ssh zrep@192.168.8.100 zfs
    bash: zfs: command not found
    ## but with the whole path it works
    zrep@fcprox:~$ ssh zrep@192.168.8.100 /sbin/zfs
    ...
    
    something is wrong, or something lacks, but what?
    that certainly has something to do with the fact that it is executed by script(?)

    maybe I found a solution here, i will try it:
    https://unix.stackexchange.com/questions/332532/how-to-set-path-when-running-a-ssh-command
     
    #8 maxprox, Nov 7, 2018
    Last edited: Nov 7, 2018
  9. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    yes, I found the solution here in Answer 2:
    https://unix.stackexchange.com/questions/332532/how-to-set-path-when-running-a-ssh-command

    "Set the PATH in the server ~/.ssh/environment" (non root users environment)
    Code:
    root@fcprox:/etc/ssh# cat /home/zrep/.ssh/environment
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:$PATH
    
    needs to be enabled by PermitUserEnvironment yes in sshd_config

    Code:
    root@fcprox:/etc/ssh# cat sshd_config
    ...
    ## wegen umgebungsvariablen fuer remote zrep u. zfs snapshots gp 20181107:
    PermitUserEnvironment yes
    ...
    
     
    #9 maxprox, Nov 7, 2018
    Last edited: Nov 7, 2018
  10. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    A problem rarely comes alone.
    After the problem with the environment variables was solved. He noticed again that rights are always missing.
    Code:
    zrep@fcprox:/root$ /usr/sbin/zrep -i rpool/data/vm-207-disk-1
    192.168.8.100 r5pool/snap_test/win7lex
    WARNING: no proper recv -o detected
    WARNING: extremely old versions of ZFS crash with volume init
    Continuing in 5 seconds....
    Setting properties on rpool/data/vm-207-disk-1
    Warning: zfs recv lacking -o readonly
    Creating readonly destination filesystem as separate step
    cannot create 'r5pool/snap_test/win7lex': permission denied
    Error: Cannot create 192.168.8.100:r5pool/snap_test/win7lex
    
    However, a simple test shows that all rights are actually set
    Code:
    ssh zrep@192.168.8.100 zfs create r5pool/snap_test/test02
    
    and with this direct command, the dataset was created on the remote backup server
     
    #10 maxprox, Nov 7, 2018
    Last edited: Nov 7, 2018
  11. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,204
    Likes Received:
    498
    you'd have to look into the zrep sources to see what exactly it is doing when running into those errors.. maybe there is some kind of verbose option to increase the output, otherwise you'd need to grep for the error messages and add some output yourself ;)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. dlimbeck

    dlimbeck Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2018
    Messages:
    137
    Likes Received:
    9
    Might be the same problem I encountered when extending pve-zsync with the --dest-user option. Does r5pool/snap_test/win7lex exist on the destination node? If so, what happens if you delete it and then try again?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  13. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    No, the destination dataset (r5pool/snap_test/win7lex) will be created by the inital (zrep -i) command
    if r5pool/snap_test/win7lex were there, it would lead to a mistake.
    I'll ask Philip Brown, who developed zrep, but I think he's sleeping on the other side of the world right now ;-)
     
  14. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    okay,
    Philip Brown wrote me to test it exactly as zrep does it. add
    -o readonly=on
    to the zfs create test

    Code:
    zrep@fcpro:~/.ssh$ zfs destroy r5pool/snap_test/rechtevorhanden
    zrep@fcpro:~/.ssh$ zfs create -o readonly=on r5pool/snap_test/test03
    cannot create 'r5pool/snap_test/test03': permission denied
    
    And with this option I get again "permission denied"
    zrep user has this rights:
    Code:
    user zrep aclinherit,clone,create,destroy,hold,mount,receive,rename,send,snapshot,userprop
    Any Ideas which permission are needed?
     
  15. dlimbeck

    dlimbeck Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2018
    Messages:
    137
    Likes Received:
    9
    In the manpage for zfs there's an entry for aclinherit which specifies that it defaults to 'restricted', maybe try 'passthrough'?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    I dont found this option 'passthrough' in the context with zfs allow in the man pages, I also get an error with this option

    Code:
    root@fcpro:/etc/cron.d# zfs allow zrep aclinherit,mount,snapshot,destroy,receive,clone,create,send,rename,hold,userprop,passthrough r5pool/zrep_test02
    cannot set permissions on 'r5pool/zrep_test02': operation not applicable to datasets of this type
    
    I will try every option for zfs allow I found there o_O
     
    #16 maxprox, Nov 7, 2018
    Last edited: Nov 7, 2018
  17. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    No, for now, I do not found a solution for the 'permission denied' message if I would create a ZFS dataset with the option readonly=on
    Code:
    zrep@fcpro:~/.ssh$ zfs destroy r5pool/snap_test/rechtevorhanden
    zrep@fcpro:~/.ssh$ zfs create -o readonly=on r5pool/snap_test/test03
    cannot create 'r5pool/snap_test/test03': permission denied
    
    I try almost every ZFS permission I found => permission denied
    after reading this: https://docs.oracle.com/cd/E23824_01/html/821-1448/gebxb.html
    I changed with chown and chmod the linux filesystem rights => permission denied
    I have looked at the direct properties of the ZFS, but found nothing
    read the ZFS allow part of this interesting presentation: http://www.allanjude.com/talks/vBSDCon2015_-_Interesting_ZFS.pdf

    => for now as a non root user I get => permission denied

    as a workaround I try to do the first clean initialization snapshot creation as root
    that seems to be working.

    because the meaning of the 'non root user' for me is only about the running cron job, in connection with the mutual passwordless ssh access and its security risk, I can live good with this workaround
     
    #17 maxprox, Nov 7, 2018
    Last edited: Nov 7, 2018
  18. dlimbeck

    dlimbeck Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2018
    Messages:
    137
    Likes Received:
    9
    Oh sorry if it wasn't clear, it's an option for aclinherit -> aclinherit=passthrough
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  19. maxprox

    maxprox Member
    Proxmox Subscriber

    Joined:
    Aug 23, 2011
    Messages:
    300
    Likes Received:
    12
    Hello dlimbeck,
    thank you, and sorry back: after I write the post I found -> aclinherit=passthrough as one of the
    direct ZFS properties, I changed it from aclinherit=restricted to aclinherit=passthrough
    but it doesn't solved my 'permission denied' problem
    But now after one first initialization replication Snapshot as root,
    the second and every normal following synchronizing work
    as a normal 'non-root' user.

    That is okay for me
    best regards,
    maxprox
     
  20. fabian

    fabian Proxmox Staff Member
    Staff Member

    Joined:
    Jan 7, 2016
    Messages:
    3,204
    Likes Received:
    498
    you need to allow each (non-user) property individually, check the man page for 'zfs allow'. you can also define a set (e.g., @zrep_properties) with all the properties that zrep touches and use that for 'zfs allow'

    edit: in case this wasn't clear, you need to do the following:
    Code:
    zfs allow zrep readonly path/to/dataset
    rinse, repeat for all other properties that zrep wants to set/inherit
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #20 fabian, Nov 9, 2018
    Last edited: Nov 9, 2018
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice