Backup CT with CEPH mount at NFS

[morfair]

Hi all!

I have PVEv4 cluster node with CephFS mount point:

192.168.x.x:/ on /pvestorage/ceph type ceph (rw,noatime,name=xxx,secret=<hidden>,acl)

On this node I have CT (lxc) in which subdir I bind-mount subdir of CephFS:

lxc.mount.entry: /pvestorage/ceph/var_flow var/flow none bind 0 0

This work ok. But I can't make backup on NFS storage named 'storage6':

INFO: starting first sync /proc/9089/root// to /mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmpINFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.entries") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.files") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.rbytes") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.rctime") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.rentries") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.rfiles") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.rsubdirs") failed: Operation not supported (95)
INFO: rsync: rsync_xal_set: lsetxattr(""/mnt/pve/storage6/dump/vzdump-lxc-171-2015_11_13-15_53_14.tmp/var/flow"","ceph.dir.subdirs") failed: Operation not supported (95)

Sorry for my English! Help pls!!

 

[tom]

seems you miss some permissions on your NFS server?

 

[morfair]

seems you miss some permissions on your NFS server?

My NFS-server fully allowed for Proxmox nodes, with PVEv3 nodes and OpenVZ CT backup it worked good!

UPD: On PVEv3 nodes I use CephFS-fuse, now I use kernel CephFS module.

 

[morfair]

Maybe I should remove -X attr from rsync? In what script it placed?

 

[wbumiller]

This is not a permission problem. NFS doesn't support xattrs or ACLs.
When you make a backup onto NFS it's best to set a large-enough local tmpdir in /etc/vzdump.conf to a directory with full support for xattrs and ACLs.
That way you also avoid sending all the data over the network several times. (There are 2 rsyncs before then archiving it with tar which needs to both read and write).

 

[morfair]

This is not a permission problem. NFS doesn't support xattrs or ACLs.
When you make a backup onto NFS it's best to set a large-enough local tmpdir in /etc/vzdump.conf to a directory with full support for xattrs and ACLs.
That way you also avoid sending all the data over the network several times. (There are 2 rsyncs before then archiving it with tar which needs to both read and write).

How to skip xattrs?

 

[morfair]

In file /usr/share/perl5/PVE/VZDump/LXC.pm in line 27 I removed '-X' attr from 'rsync' and backup began to work well.

 

[wbumiller]

Currently we have no setting for that, you'd have to adapt the rsync call in /usr/share/perl5/PVE/VZDump/LXC.pm
Though I see that these seem to be ceph specific xattrs and it probably makes sense to restrict rsync to only copying the user namespace (plus `security.capability`) as that's what the final archive would contain.