Docker support in Proxmox

sorry that I reopened this old discussion with my comment(s). People still seem to have strong opinions about how containers fit in their infrastructure. Didn’t mean to start a religious discussion about it.

To conclude from my side, I really like the overall direction Proxmox is taking in recent times (firewall, cloudinit, etc. But also the new backup server!). I think many people are using Proxmox as a lightweight private cloud these days. At least it prevents me from running a fat, complex OpenStack installation. I only wish the cloudinit support would be more complete (set userdata in the web interface, etc) and there was officially supported Terraform support.

Regarding containers: On any public cloud OCI/Docker containers run in VMs (alone for security reasons). So what Proxmox lacks a bit here is orchestration/management of that. I would wish for better support here (similar to the direction VMware is taking with their Kubernetes features), but I can understand if it’s out of scope.

But better cloudinit support would be really important imho so that Proxmox can at least fully serve a AWS EC2 like usecase and users can then run Kubernetes distros on top. Things are close, but not yet fully there.
 
  • Like
Reactions: Alibek
sorry that I reopened this old discussion with my comment(s). People still seem to have strong opinions about how containers fit in their infrastructure. Didn’t mean to start a religious discussion about it.

To conclude from my side, I really like the overall direction Proxmox is taking in recent times (firewall, cloudinit, etc. But also the new backup server!). I think many people are using Proxmox as a lightweight private cloud these days. At least it prevents me from running a fat, complex OpenStack installation. I only wish the cloudinit support would be more complete (set userdata in the web interface, etc) and there was officially supported Terraform support.

Regarding containers: On any public cloud OCI/Docker containers run in VMs (alone for security reasons). So what Proxmox lacks a bit here is orchestration/management of that. I would wish for better support here (similar to the direction VMware is taking with their Kubernetes features), but I can understand if it’s out of scope.

But better cloudinit support would be really important imho so that Proxmox can at least fully serve a AWS EC2 like usecase and users can then run Kubernetes distros on top. Things are close, but not yet fully there.
That's exactly the thing, I would also love more cloud-init capabilities in terms of user-data directly setable in the WebUI as well as feature-parity with the cloud-init implementation in Openstack.

Reason: many KVM images with cloud-init support only work well using Openstack, with Cloudstack or any other KVM-based infra it's often a workaround-mess to get the same functionality.
 
  • Like
Reactions: Alibek
Very old thread and it's basically dead, so i don't need to reply to anyone:)

But with openzfs 2.2, docker will be working flawless inside lxc containers.
I'm preparing a test right now, just have some troubles atm compiling the kernel with zfs for Proxmox. Basically im trying to compile the 6.3 kernel with openzfs 2.2rc1.

However long story short, i think that running docker/kubernetes/containerd is anyway better suited inside an lxc containers.
I mean you can do backups and migrate them etc...
Imagine how much work that would be for the Proxmox devs, to implement native docker support that is cluster"izable" :)

However, i would rather suggest to implement lxd, to proxmox.
Lxd utilizes lxc at the same way as Proxmox does, but it expands lxc containers to be clusterizable.
Means allows for proper HA, without the shutdown/migrate/restart route + some other nice features.
Lxd utilizes kvm either, which is useless to us, but it adds at least mainly a lot features to lxc.

Especially since docker won't be an issue anymore on lxc containers with openzfs.

Btw, im running docker containers on Unprivileged lxc containers on ext4 backend storage since almost forever, and it worked always very perfect.

Lately im running even docker containers inside unprivileged lxc container with overlay2 and nesting only on zfs.
And it works with 98% of the containers amazingly well either.
I stumbled only over 2 docker images so far, that didn't worked with this method, one of them is speedtest-tracker.

However, it's a suggestion about lxd. So if it comes or not, doesn't matter for me very much either, since im extremely happy with Proxmox anyway.

Cheers
 
Using with success docker inside unprivileged lxc on top of ceph. Nothing too complex but never had an issue. As I wrote in this thread some time ago, + 1 to not use docker directly un proxmox host.
 
That's great news. I've been wanting this for years after experiencing zfs delegation in jails with FreeBSD.
Very similar situation here. Just to add to the previous comment, openzfs2.2 does multiple things:
  • OverlayFS (`overlay2`) is functional on top of zfs 2.2
    • This will hopefully come relatively soon, and should allow the default docker install (which uses overlay2) to function in an LXC container stored on a ZFS filesystem, no ext4 zvols required
    • Bonus tip: You might need to update your pool to a new zfs version for it to work properly
  • Support for ZFS managment within containers
    • I suspect this will take longer to be integrated with LXC and by proxy Proxmox - but from a quick look at the patch notes ("...added namespace delegation support for containers"), it seems like there are a set of features that would enable containers (LXC, perhaps even docker?) to manage a ZFS pool (or a subset of it, like a dataset/subvol - not exactly sure on the details there). I'm not familiar with BSD Jails, but it sounds quite similar.
I might create a separate forum post about this, but a note for the Proxmox Devs:
Like I mentioned above, it would be nice if the pve version of zfs 2.2 could be released sooner rather than later - It would be nice to have `overlay2` support at the least even if proper support for namespace delegation (among other things) takes longer to implement.

Edit:
Found this thread after posting: https://forum.proxmox.com/threads/o...-2-for-proxmox-ve-8-available-on-test.135635/
I'm not sure when it will be available in the `pve-no-subscription` repository, but it's available in the testing repository.
 
Last edited:
Very similar situation here. Just to add to the previous comment, openzfs2.2 does multiple things:
  • OverlayFS (`overlay2`) is functional on top of zfs 2.2
    • This will hopefully come relatively soon, and should allow the default docker install (which uses overlay2) to function in an LXC container stored on a ZFS filesystem, no ext4 zvols required
    • Bonus tip: You might need to update your pool to a new zfs version for it to work properly
  • Support for ZFS managment within containers
    • I suspect this will take longer to be integrated with LXC and by proxy Proxmox - but from a quick look at the patch notes ("...added namespace delegation support for containers"), it seems like there are a set of features that would enable containers (LXC, perhaps even docker?) to manage a ZFS pool (or a subset of it, like a dataset/subvol - not exactly sure on the details there). I'm not familiar with BSD Jails, but it sounds quite similar.
I might create a separate forum post about this, but a note for the Proxmox Devs:
Like I mentioned above, it would be nice if the pve version of zfs 2.2 could be released sooner rather than later - It would be nice to have `overlay2` support at the least even if proper support for namespace delegation (among other things) takes longer to implement.
Pve kernel 6.5 with openzfs 2.2 is released.
And all the docker issues are solved, tested here already.
 
  • Like
Reactions: SInisterPisces
Pve kernel 6.5 with openzfs 2.2 is released.
And all the docker issues are solved, tested here already.
Sort of - it requires the `pvetest` repository, it hasn't been released to the "stable" yet. Updated the original post.
 
Sort of - it requires the `pvetest` repository, it hasn't been released to the "stable" yet. Updated the original post.
If i can make you feel better, im running that kernel/openzfs test already on 5 servers here, its perfect :)
 
Pve kernel 6.5 with openzfs 2.2 is released.
And all the docker issues are solved, tested here already.
If we're setting up a new CT to run Docker, do we need to do anything specific to the CT itself to get things working properly? Or, do we just install and use Docker as if it were bare metal?

I'm assuming I'll need to upgrade my zpool(s). I've never had to do that before. A bit scary.
 
If we're setting up a new CT to run Docker, do we need to do anything specific to the CT itself to get things working properly? Or, do we just install and use Docker as if it were bare metal?

I'm assuming I'll need to upgrade my zpool(s). I've never had to do that before. A bit scary.
Yeah you need to upgrade your zpools.
But i did that already on ton of systems, almost all in live environments.
The upgrade takes like a millisecond and you don't need to shutdown any vm on the running pool or anything.

About upgrading your pools i wouldn't worry even a second.
Only downgrades are impossible, means your pools won't work anymore with anything below zfs 2.2.

Otherwise im running docker almost always on unprivileged lxc containers without anything special in them.
Just uninstalling apparmor always in all my lxc containers, cause apparmor is simply stupid.
The idea is great but they didn't thought to make the apparmor configuration easy.

On privileged lxc containers docker runs without apparmor great either.

Cheers :)
 
Yeah you need to upgrade your zpools.
But i did that already on ton of systems, almost all in live environments.
The upgrade takes like a millisecond and you don't need to shutdown any vm on the running pool or anything.

About upgrading your pools i wouldn't worry even a second.
Only downgrades are impossible, means your pools won't work anymore with anything below zfs 2.2.

Otherwise im running docker almost always on unprivileged lxc containers without anything special in them.
Just uninstalling apparmor always in all my lxc containers, cause apparmor is simply stupid.
The idea is great but they didn't thought to make the apparmor configuration easy.

On privileged lxc containers docker runs without apparmor great either.

Cheers :)

Thanks! If you were building an LXC from scratch to work as a Docker host (or a VM, I guess), which distro would you use?

I'm planning to use the Debian LXC template; I prefer Ubuntu but don't like that I can't uninstall snapd anymore. I could just disable it, but it makes me wonder what they'll force on users in the future.
 
Thanks! If you were building an LXC from scratch to work as a Docker host (or a VM, I guess), which distro would you use?

I'm planning to use the Debian LXC template; I prefer Ubuntu but don't like that I can't uninstall snapd anymore. I could just disable it, but it makes me wonder what they'll force on users in the future.
I personally would use either alpine or NixOS. NixOS is nice since you can manage the entire LXC configuration declaratively, while Alpine is good because it is so much more lightweight than pretty much anything out there. You need to enable the alpine edge repository to install a recent version of docker, however.
 
  • Like
Reactions: SInisterPisces

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!