Help with NFS mounting on host

[LordRatner]

This might be more of a Linux question, but here it goes:

I have two nodes. Node1 and Node2.

Node2 has a secondary zfs pool that serves as the bulk storage. In order to make this storage accessible to an LXC regardless of the node that it is on:

Node1 has a /mnt/nas folder mounted using nfs to the pool on node2, using node2's LAN ip address. NFS is done via autofs.

Node2 has a /mnt/nas folder mounted through the same mechanism, but to the localhost ip. This way the lxc looks for the same mnt folder regardless of what node it is on. The mount is done through a lxc.mount.entry line in the conf since this allows for snapshots and migration.

The whole thing is working really well except for one quirk... anything created on the nas pool (which is on node2) by an lxc on node1 is done by nobody:nogroup

But anything created by an lxc on node2 is created as 100000:100000. For most applications this doesn't seem to matter, but Nextcloud is unhappy with it.

I'm sure this is happening because even though the LXC is accessing the storage through an nfs mount, the host clearly knows it is accessing itself and is applying a different user to the files created by LXCs. I've tried changing the autofs configuration on node2 to use it's LAN ip instead of 127.0.0.1, but it made no difference.

So how can I force the nfs mount to use nobody:nogroup even though it is mounting to itself? I suppose the alternative would be fine too, if both used 100000:100000 (or any number, as long as they were the same). I just need them to be the same regardless of which node the lxc is on at the time.

Thanks!
Seth

 

[BobhWasatch]

You could esport the share with all_squash.

 

[LordRatner]

You could esport the share with all_squash.

I've got all_squash on the shares, and it works for the node1-->node2 share, but not the node2-->node2 share. I suspect because nfs is being bypassed.

 

[BobhWasatch]

Perhaps an nfs mount on localhost gets turned into a bind mount? Anyway, I guess the next thing would be to remove all_squash and make sure that both LXC use the same UID mappings.

 

[LordRatner]

Perhaps an nfs mount on localhost gets turned into a bind mount? Anyway, I guess the next thing would be to remove all_squash and make sure that both LXC use the same UID mappings.

That's what I think is happening. There's a "nobind" option in the manual for auto.master that is supposed to fix this issue, but it's not working for me.

 

[BobhWasatch]

Are you using direct maps? There are some restrictions on nobind in that case. Another option might be to directly mount the share on node2 rather than using autofs.

 

[LordRatner]

Are you using direct maps? There are some restrictions on nobind in that case. Another option might be to directly mount the share on node2 rather than using autofs.

I'm not positive what a direct map is. My exact setup is spelled out here: https://stackoverflow.com/questions/77720431/using-autofs-to-mount-a-local-directory

I can make it work by forcing the node1 --> node2 connection to use 100000:100000, but that means all the LXCs on node2 will have to be using their root user to match up with that. Ideally I'd rather have everyone just using nobody:nogroup

I really appreciate your help.