Datasets as datastore for docker container?

[oldchiefphil]

Hi everyone,

I’m new to selfhosting and also new to Proxmox and just taking my first baby steps with home labbing. My server is currently running several VMs, and I have a few questions regarding storage configuration.

I created a zpool called vm-pool where all my VMs reside (Proxmox itself is installed on a separate ZFS - root-pool).

Recently, I created a VM where I plan to install Docker.

1. How do I determine the appropriate disk space for this Docker VM?
   Since I expect the container’s storage needs to grow over time (one docker-container will be immich), I’m considering creating a directory on my ZFS vm-pool and mounting it into the Docker container. After some research, I learned about ZFS datasets—wouldn’t this be exactly what I need? For example, I could create a dataset called docker-data and mount it inside the Docker container. This way, I’d allocate only a few GB for the VM’s host system while all the Docker data would reside in this dataset, which can be as large as the entire ZFS pool and I do not have to bother with resizing the disksize in the vm.
   
   
2. Why do I have to create datasets via the CLI?
   I noticed that Proxmox’s GUI doesn’t offer this functionality. Is it not built in yet?
3. Should I store each VMs also in their own datasets?
   Currently, all VMs reside in the root of the vm-pool. Would it be better to place each VM in its own dataset?

The next big part will be backups. But I think, that is another story.

Thanks in advance for any guidance!

 

[dj423]

I imagine you can, but keep in mind if you are going to mount a volume on the host to a container - will that dataset be portable to other hosts? Won't your docker VM be married to that host? You won't be able to migrate to other hosts down the road as you expand. - I don't know your use case or any of that, but I try to keep my VM's and containers as agile as I can, so they can be moved around during maintenance work, and for my use case keeping all my instances on a shared storage array works out pretty well.

But I'm sure you can hard mount your containers to zfs datasets, but I don't really know of much utility in practice. I keep my workloads within the container/VM, then just keep those instances on reliable, fast storage that is shared between hosts, so they can be moved around as needed. Now I could totally be wrong about zfs datasets and their ability to be shared, I just don't know of a use case where I would want to mount to a container as persistent storage?

 

[oldchiefphil]

The idea behind my question is as follows. When I create a Docker VM—for example, using Debian—I need to allocate a fixed amount of disk space for the VM, say 100GB. However, when I start using cloud services or image hosting, I eventually risk running out of disk space. Although I can resize the disk in Proxmox, this requires me to expand the disk within the VM by moving or deleting partitions and then recreating them.

My idea is to allocate enough space within the VM to install all my Docker containers, while outsourcing their data to a separate dataset that can grow as large as my zpool allows. This approach would eliminate the need to constantly resize the VM when disk space becomes limited.

 

[dj423]

As a general rule I set my smaller workload VM's to 60GB or so, and larger ones about 120G. I try to keep things as small as possible to maintain portability. For applications that need lots as in several terabytes of storage (media, image libraries, etc.) I use NFS shares to larger storage pools.

For your image server, you can try and total the current storage need for your library, then double that amount to give you room for growth.