Docker support in Proxmox

laowolf

Member
Jul 29, 2014
66
1
8
Just to continue this discussion for intellectual purposes....

Whatever we think of LXC vs Docker - the fact of the matter is that the industry has chosen Docker for most workloads.

Secondly, kubernetes is the golden nugget but the space where we need a tool is to compete is with openshift which is basically a massive wrapper around kubernetes and adds the "missing" associated services.

openshift is centos based yuck.... We need a debian based kubernetes wrapper
Actually, Docker is not Standard. RunC is the industry standard.
 

Telekomiker

Member
Dec 2, 2012
33
0
6
close to Frankfurt / Germany
Hi to all !

however the discussion runs:

it would by nice to start a docker container means the application based of this contailer without any additional work and knowledge for USERS like me.

That's all.

Best regards from Germany

Telekomiker !
 

LnxBil

Famous Member
Feb 21, 2015
4,327
428
103
Germany
Developers industry. IMHO.
Isn't there integration in WS2016 and VMware? Recently, we got our first customer asking about setting up a docker environment and he is not a developer and we not an infrastructure provider.
 

gosha

Active Member
Oct 20, 2014
275
18
38
Russia
Isn't there integration in WS2016 and VMware? Recently, we got our first customer asking about setting up a docker environment and he is not a developer and we not an infrastructure provider.
Isolated cases determines the industry? :)
Let's wait for the massive Docker use outside the developer environment...
 

LnxBil

Famous Member
Feb 21, 2015
4,327
428
103
Germany
If you believe in the cloud providers, there are millions of containers out there. Obviously, not all on docker, but similar techniques. Biggest are AWS and nowadays also Azure. Maybe the goals of Proxmox VE and a real PaaS-Cloud with Docker are just too different, maybe even disjoint.
 

LnxBil

Famous Member
Feb 21, 2015
4,327
428
103
Germany
I assume you know that (almost) everything at Google runs in containers, right?
Oh yeah ... I forgot Google and their kubernetes stuff. The industry in currently in a big change towards continuous deployment, so there is no need for "old" virtualization techniques besides providing the infrastructure. That is going to be a big game changer for the administrators, because they do not need to prepare machines etc, the developers can just deploy them and everything works. It's really amazing how simple that is at the moment.
 

morph027

Well-Known Member
Mar 22, 2013
426
53
48
Leipzig
morph027.gitlab.io
Just picking up...As said before, Docker stuff would be great, but it will take lots of time to develop and test which i think is better spent on other tasks. It's not VMware having huge amount of developers, also the PVE team does a great job not only providing a hypervisor, just think of all the storage possibilities. Would be great if there are resources, but can live without.

Last week i've got some spare time to check if and how we can use some "cloud service" internally. Ended up using each platform for what it does best. Proxmox for KVMs, which are part of a Docker Swarm. Setting up using ansible and my "kickseed" stuff with some services (Selenium Grid, CI Runners, ...) took my a day, finalizing another one, documentation and creating a workshop for our developers probably whole next week :D

Things i've used:

- Proxmox of course!
- https://docs.ansible.com/ansible/proxmox_kvm_module.html + Preseed to create Swarm Nodes
- Docker Swarm
- https://github.com/ManoMarks/docker-swarm-visualizer/
- http://portainer.io/
 
  • Like
Reactions: fleischkarussel

morph027

Well-Known Member
Mar 22, 2013
426
53
48
Leipzig
morph027.gitlab.io
The whole setup was blazing fast with our CI tests, devs are loving me right now ;) No problem there with stateless containers, in fact this is desirable to have a clean state on each CI run.

For production services, i'm trying the same just backed with some persistent storage. Guess it will be Ceph, as it's already builtin ;)

Thinking of self-healing production services makes me smile...
 

LnxBil

Famous Member
Feb 21, 2015
4,327
428
103
Germany
I also investigated docker-machine and played around a little with their drivers. Maybe there is someone willing to create a Go driver for Proxmox VE such that you can use the Proxmox VE IaaS to deploy docker fully-automatic with docker-machine. A simple working example should not be too hard if you're familiar with Go. I tried, but I do not have time right now to learn go and get it to work. I played around with the proxmox-go library and got simple authentication working. If someone would be interested in programming it, I can talk you through - just contact me.

That would IMHO be the best way to deploy Docker (PaaS) on Proxmox VE (IaaS) - use each technology for what they were created.
 

shantanu

Active Member
Mar 30, 2012
104
6
38
I have been experimenting with nomad+consul to create VMs which are able to create dockers.
A two tier architecture;
* one set of nomad+consul monitors the VMs ("qm agent NNN network-get-interfaces" as a health check to mark machine is ONLINE)
* second set of nomad+consul for the dockers created inside each VM (as per requirement)
these dockers which may scale out/scale in can be reached from outside using a proxy like "gobetween" or "haproxy+consul-template"


This is in a very (very very) basic form and the setup itself is not automated.
I may have more when I am able to "automize" the end-to-end flow for setting up a new PVE server to creating VMs as well.

Regards,
Shantanu
 

morph027

Well-Known Member
Mar 22, 2013
426
53
48
Leipzig
morph027.gitlab.io
Right now, there's a huge gap (and need) for elastic persistent storage. Here's a list of current available volume plugins:

https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins

However it is not up to date yet. I don't like the idea of block storage attached to containers, as it's mor work to extend/shrink compared to file systems. My lab system right now is running GlusterFS (withoug plugin unfortunately, just mounts on all swarm nodes). Currently, Minio is a rising star, easy to use, easy to scale, cluster/cloud in mind. As it's S3 and defacto standard, i would love if storage becomes more application centric instead of infrastructure read have the app talking to S3 directly instead of letting the OS take care of.
 

zeigerpuppy

New Member
Nov 19, 2017
2
0
1
118
a relatively straightforward way to implement docker support would be to use clear containers.

see https_clearlinux.org/features/clear-containers

These already use KVM to provide docker containers in a virtualized environment and also support management with kubernetes.

In the next month or so I'll try setting this up on a proxmox base.
 

zeigerpuppy

New Member
Nov 19, 2017
2
0
1
118
clear containers have a few dvantages, as far as I can see,
  1. they are faster to boot as the vm is stripped down
  2. they support virtFS
  3. they have an inbuild management interface that can talk to kubernetes
For deployment purposes, tjis allows spinning up and down docker instances in a managed fashion. It also allows load balancing and network autodiscovery which is hard to achieve with a standard vm.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!