TrueNAS VM, autofs mount, dependent LXCs?

[wespiard]

I'm running Proxmox VE 8.2.7. I have a TrueNAS Core VM, and a few of my unprivileged LXCs access an NFS share provided by the TrueNAS VM via a bind mount.

The solution I ended up with is using autofs to mount the TrueNAS VM's share to the Proxmox host and then bind mount that share to the unprivileged LXCs. To get this to work, I have to use a startup delay of 30-60 seconds after the VM starts, or the LXCs will fail to boot (since the bind mount directory doesn't exist instantaneously).

I'm wondering the following: is there is a way to do this without using autofs?

EDIT: See comments below. Now I'm using an /etc/fstab entry with the x-systemd.automount option, which removes the need to install autofs. A startup delay after the TrueNAS VM boots is still required, though.

 

[BobhWasatch]

Maybe setting the start order of the TrueNAS to be earlier than the CT's that need it. You can also add a delay before the startup moves on to the next VM/CT.

ETA: I use autofs for this as well, although I have no bind mounts. I just make sure that the NFS VM is started before any of the ones that depend on it and it works fine.

 

[wespiard]

Maybe setting the start order of the TrueNAS to be earlier than the CT's that need it. You can also add a delay before the startup moves on to the next VM/CT.

ETA: I use autofs for this as well, although I have no bind mounts. I just make sure that the NFS VM is started before any of the ones that depend on it and it works fine.

As a matter of fact, that's what I'm doing:

To get this to work, I have to use a startup delay of 30-60 seconds after the VM starts

I only chose bind mounts so that I didn't have to install autofs inside the LXCs.

 

[kaivalagi]

I have added mounts for my proxmox node via storage>smb (but nfs would be an alt) which mounts to a local /mnt/pvr/<smbshare> folder. This is then bind mounted for an unprivileged lxc to use, much like you describe, but right now I manually start up the lxc.

The only way I see a solution for the mounts after initial bootup is to have a script running on the host during startup checking for the truenas share availability which then starts the lxc's via the cli once all is well...I'll be playing with this tomorrow and will feed back if I get anywhere.

Something like this for nfs:

#!/bin/sh
while! showmount -e server | grep -q /share;
  do  sleep 1
done
echo "NFS share is available, starting dependant lxc's"
pct start 100

can you elaborate more on the autofs approach you've taken in case that could work as part of a solution too?

edit: decided to trial TrueNAS 24.10 as the host, rather than a VM with hdd passthrough, for this nas setup I'm working on, using docker instead of lxc's will hopefully be fine...proxmox on hold, maybe moved to the next project when I get to my gaming pc...usb passthrough for my vr headset will be interesting haha. Hope the above helps.

 

[wespiard]

I have added mounts for my proxmox node via storage>smb (but nfs would be an alt) which mounts to a local /mnt/pvr/<smbshare> folder. This is then bind mounted for an unprivileged lxc to use, much like you describe, but right now I manually start up the lxc.

The only way I see a solution for the mounts after initial bootup is to have a script running on the host during startup checking for the truenas share availability which then starts the lxc's via the cli once all is well...I'll be playing with this tomorrow and will feed back if I get anywhere.

Something like this for nfs:

#!/bin/sh
while! showmount -e server | grep -q /share;
  do  sleep 1
done
echo "NFS share is available, starting dependant lxc's"
pct start 100

can you elaborate more on the autofs approach you've taken in case that could work as part of a solution too?

edit: decided to trial TrueNAS 24.10 as the host, rather than a VM with hdd passthrough, for this nas setup I'm working on, using docker instead of lxc's will hopefully be fine...proxmox on hold, maybe moved to the next project when I get to my gaming pc...usb passthrough for my vr headset will be interesting haha. Hope the above helps.

Were you using a truenas VM inside of proxmox to host the smb shares? If so, then setting the truenas VM to boot first, with a 30-60 second delay before the LXCs would have potentially worked. I may be misunderstanding your setup, though.

Even if you don't end up using this, I'll still post in case anyone finds via Google in the future. Here's what I did (to the best of my memory):

• Install autofs on proxmox host
• Append mount point to /etc/auto.master config file, e.g.,
  /mnt/nfs /etc/auto.nfs --ghost --timeout=60
• Add the shares you want to /etc/auto.nfs, for me it's just one right now, e.g.,
  data -fstype=nfs,nolock,rw 192.168.0.209:/mnt/truenas_pool/data
• Depending on if the daemon is started or not, you may need to run systemctl start autofs or systemctl restart autofs
• In my example, the mount should show up at /mnt/nfs/data based on the naming I've used.

To make sure any LXCs dependent on this share don't try to boot before my truenas VM has booted, I do the following:

• In the Proxmox GUI datacenter view, click on the truenas VM and go to options > start/shutdown order
• Set order to 1 (so that it is the first to boot)
• Set startup delay to 45 (you may need to adjust this based on how long the VM takes to start and ready the shares)
• Similarly, for each LXC that is dependent on the share, change its startup order to "2".

So what ends up happening when proxmox boots is autofs prepares the mount directory (even though the share isn't available yet since the truenas VM hasn't booted yet), and then 45 seconds after the VM is up, the LXCs boot.

This also affects the shutdown order, I believe. So the LXCs will shutdown first. I think I had some issues in the past where the VM would try to shutdown first but it hung (maybe because the LXCs were still using the shares?), but I'm not sure exactly.

I haven't had any startup/shutdown issues since using this setup.

 

[wespiard]

After some more research, I tested using the /etc/fstab systemd.automount option and that seems to work as well, without needing to install and configure autofs. That seems to be working as well, so I'll probably switch to that so I don't have to worry about autofs for future installs.

Entry in /etc/fstab:
192.168.0.209:/mnt/truenas_pool/data /mnt/nfs/data nfs x-systemd.automount,x-systemd.mount-timeout=10 0 0

 

[wespiard]

Maybe setting the start order of the TrueNAS to be earlier than the CT's that need it. You can also add a delay before the startup moves on to the next VM/CT.

ETA: I use autofs for this as well, although I have no bind mounts. I just make sure that the NFS VM is started before any of the ones that depend on it and it works fine.

My previous comment may be useful to you.

You may not need to install autofs on your LXCs, and instead could use an /etc/fstab entry with the x-systemd.automount option.

Then it's a matter of preference: configuring the bind mount for each LXC or configuring /etc/fstab for each LXC. I am not sure if there would be any performance difference (not an expert).

 

[kaivalagi]

Well I ordered a hba so proxmox hosting truenas is back on the cards For now I've passed through a hard drive to a truenas vm and I've setup autofs and configured delays where needed and all is well...I'll try the systemd approach later maybe

Why were you looking for an autofs alternative other than to remove a package dependency? this approach seems really nice, especially for me with so many shares and using a file per nas etc etc

 

[wespiard]

Why were you looking for an autofs alternative other than to remove a package dependency? this approach seems really nice, especially for me with so many shares and using a file per nas etc etc

No major reason, honestly. Just didn't want to have the extra dependency, and the need to configure more files (like auto.master, etc.). I just like the idea of it being contained only in the fstab file and using built-in functionality.

 

[kaivalagi]

does the automount stuff work with samba too do you know? is it just a retry mechanism for normal fstab entries through adding some extra arguments in effectively? I'll do a bit of reading in a bit

 

[wespiard]

does the automount stuff work with samba too do you know? is it just a retry mechanism for normal fstab entries through adding some extra arguments in effectively? I'll do a bit of reading in a bit

Can't say for sure since I don't have any samba shares, but I've seen people using fstab for samba shares. So I'd imagine it's possible to add the automount option just like nfs.