add storage accessible for some VMs (linux/win)

[mediaklan]

Hi to you all !
I have 3 hdd on my system : 1 nvme for proxmox itself, another ssd for VM/CT storage, and another one that I would like to use as a data disk (don't know yet if I'm gonna use the whole disk), somewhat accessible for both some windows / linux VMs.

How would you go for that configuration ? Any idea/advices ?
Thank you !

 

[Dunuin]

There are only 3 way to bring the disk into the VM:
1.) you create partitions and use "qm set" to pass the partitions (as a virtual disk) into a VM. Worst way in my opinion.
2.) you format the disk with ZFS + sparse or LVM-thin and use it as a VM storage too. Then you can create virtual disks on it and attach them to the VMs. Benefit compared to option 1 would be that you waste less space because you can use thin-provisioning.
3.) you format the disk with a filesystem of your choice, turn your server into some kind of NAS and use SMB shares inside your VMs. This would be the only way how different VMs could access the same data at the same time (shared folder).
3.a.) For a NAS you could install a SMB server directly to your PVE host (without any GUI and generally you don't want to run stuff like that directly on your hypervisor)
3.b.) or you could create a OpenMediaVault LXC and bind-mount the filesystem of a local disk into the LXC so the LXC could act as a NAS.
3.c.) If you don't like OpenMediaVault or LXCs you could also create a NAS VM, give that VM a virtual disk that uses the complete physical disk, then create a filesystem of your choice inside the NAS VM on that virtual disk and let the NAS VM act as a SMB server.

I would prefer option 3b.

 

[mediaklan]

Thank you a lot Dunuin ! After some thoughs, I was leaning toward option 3, specifically 3a/3c (because I didn't even thought about 3b at all, thank you for that !).
Option 3, well - 3b - is clearly the best option for me (if i recall correctly, LXC is more lightweight than a full VM), plus that gives me the opportunity to learn something new in the process (new to proxmox, so I need to learn more about LXC anyway).
No problem for OpenMediaVault LXC ? I think I might have seen people talk about issue with OpenMediaVault LXC ?

 

[Dunuin]

It isn't easy but should work. Did the same some months ago.

You need to:
1.) create a unprivileged LXC ("debian-10-standard" as template) stored on your VM storage (1 vCPU, 512MB RAM, 256MB swap should be fine for a first test)
2.) run the OMV installer script inside your LXC to turn that Debian 10 LXC into a OMV 5 LXC
3.) create a empty folder on your host as a mountpoint for the new disk (for example mkdir /mnt/data && chown root:root /mnt/data)
4.) partition your data disk (for example using fdisk or parted commands on your host) to have a single partition
5.) format your data disks partition with a linux filesystem of your choice (for example something like mkfs.xfs /dev/yourDeviceAndPartition on your host)
6.) make that partition automatically mount at boot to the folder created in step 2 by adding a new line in your hosts "/etc/fstab"
7.) manually mount it by running mount -a...you now should have a working XFS partition mounted at "/mnt/data"...but first verify that it works
8.) on the host, create a new folder on that partition that you want to pass to the LXC so the LXC can use it to store the SMB shares there (for example run mkdir /mnt/data/omv)
9.) inside the LXC, create a new folder where you want your folder from the last step bind-mounted to (for example run mkdir /media/omv)
10.) on your host edit the LXCs config file (for examlpe nano /etc/pve/lxc/100.conf if your LXC would have the VMID 100) and add a new line there to create a bind-mount. Such a line could look like this mp0: /mnt/data/omv,mp=/media/omv,mountoptions=noatime and would bind-mount the folder "/mnt/data/omv" on your host" to the folder "/media/omv" inside your LXC.
11.) now comes the most complicated part. Unprivileged LXCs use user and group remapping so users/groups you see inside the LXC with a UID/GID of 0 to 65535 will be using the UID/GID of 100000 to 165535 on th host. Lets say you omv folder is owned by root (UID 0) on the host and you bind-mount into the LXC. Your root inside the LXC can't access that folder because the LXCs root is reported as UID 0 inside the LXC but is actually internally using the UID 100000. So to prevent right problems you got 3 options:
11.a) make the hosts omv folder read/write/executable by everyone (chmod -R 777 /mnt/data) which isn't great for security
11.b) change the owner of the hosts omv folder to the mapped UID of the user your OMV is running as inside the LXC (so if your OMV is running as the LXCs root user you make that hosts folder owned by UID/GUI 100000 like this: chown -R 100000:100000 /mnt/data)
11.c) you manually edit the user/group remapping like described here so the user that ownes the folder you want to bind-mount won't be remapped so the UID/GID on the host and inside the LXC would be identical
12.) you now should have a folder "/media/omv" inside your LXC which you can use to store your SMB/NFS shares and so on. Everything written to "/media/omv" inside your LXC will end up on the host in the folder "/mount/data/lxc" which again will be stored on your data disk.
13.) you can then use the OMVs webUI to create your SMB shares, manage rights and so on

 

[mediaklan]

oO'
That ... is an awesome answer and guide.
A big thank you for your help in my project. Part 11 would have certainly proven to be a headache without your advices and guide here.
'Seems I have a lot of things to do and prepare for now ^^'

Thank you again !

 

[mediaklan]

hmm... It seems I'm missing something. Everything looks like it's supposed to work, but it's not.
What I did :
- step 1 to 10 : OK
- in lxc console, I can see the bind-mount :
/dev/sdb1 xfs 1.9T 14G 1.9T 1% /media/omv
- I choose method 11b (chown -R 100000:100000 /media/omv) : OK. I can read/write a file from the host and read/write in the lxc, and vice-versa.
- I don't see any disks or mount points (not even the system disk) in OMV.
While not sure if it's related, trying to "scan" in Storage/disks throws this error at me :

Error #0:
OMV\ExecException: Failed to execute command '[ -x /sys/class/scsi_host ] && for hostdir in $(find /sys/class/scsi_host -iname "host*" -type l); do echo "- - -" > ${hostdir}/scan; done' with exit code '2':  in /usr/share/php/openmediavault/system/shellscript.inc:52
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/diskmgmt.inc(322): OMV\System\ShellScript->execute()
#1 [internal function]: Engined\Rpc\DiskMgmt->rescan(NULL, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('rescan', NULL, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('DiskMgmt', 'rescan', NULL, Array, 1)
#5 {main}

- -
Any idea please where I might have gone wrong ?
Thank you for your help !

 

[Dunuin]

Bind-mounts are just folders and no virtual disks. So it won't show up as a drive. If you create a SMB/NFS share you just point OMV to the path of the bind-mount.

 

[empty.dates]

Bind-nounts are just folders and no virtual disks. So it won't show up as a drive. If you create a SMB/NFS share you just point OMV to the path of the bind-mount.

Hi,
Thank you for awesome manual, mounting working just fine betwen proxmox and LXC. Installation by script also went well.
but I still cant add any shared folder.

OMV rootfs plugin installation finished with "No root filesystem has been found."
And when I am trying to add any share folder it keep asking for device and do not allow me type anything (and drop down is empty)


I was also doing "chown -R 100000:100000 ..."

Could you please advise what else can I try?

EDITED: actuall - what I am looking for - is a lightweight file sharing possibilities from proxmox host drive to home network (literally SMB+GUI for right managment) and probably alpine + bind-mount + samba + webmin is better option here. UI is not nice but it works (in comparison to OMV ) And because I trust to my hardware raid based on H700 - I dont need NAS monsters to keep eye on my disk as it hidden in any cases.