Best way to use Docker?

[cmer]

What's the best way to use Docker containers on PVE? I assume I should create a VM and run the containers inside it? Or is there a way to run them directly on top of the host?

 

[morph027]

We are running a lots of VMs for Docker Swarm.

 

[dietmar]

What's the best way to use Docker containers on PVE? I assume I should create a VM and run the containers inside it?

yes

 

[engin]

@morph027 are you using kvm or lxc for the swarm clusters?

 

[morph027]

KVM. I've also tested Docker native on PVE, but this was not reliable.

 

[engin]

@morph027 are you using zfs for the underlying storage?

i'm in the middle of preps for the swarm & k8s cluster; my nodes have 3 ssd with zfs raidz1

and compression lz4 as well.

if you're have any test /measurement, using zfs, any suggestion for the tuning?

 

[morph027]

Jip, loving ZFS since ever, glad it came to Linux and PVE

We mix some local VMs on RAIDZ1 SSDs with sheepdog clustered VMs (also on top of ZFS datasets). It performs awesome so i did not invest any further time in fine tuning. If you're interested in special numbers, i can test them.

We are running Swarm on top of the VMs.

 

[iconvergence]

vm with kvm
storage with ceph
running thousand of containers
using ansible for manage the vm
using rancher for container orchestrator

 

[engin]

what about ingress traffic, do you guys have any automation for it?

let's say we have 5 proxmox nodes with public network
container vm nodes (the kvm's) running on private lan

may be nginx / haproxy on proxmox nodes?
lets-encrypt certificate automation?

thanks for the feedbacks

 

[morph027]

Inside Docker, we either use docker-flow-proxy or traefik (can handle letsencrypt) ...

Outside we do have simple haproxy's with failover IP address.