Installing Rancher Kubernetes on LXC

[Dendi Suhubdy]

Guys does anyone know what it takes to deploy and run rancher kubernetes on an LXC container? I'm sure I ran this tutorial https://github.com/polinchw/rancher-kubernetes-on-bare-metal correctly but it seems that when accessing the public IP the rancher interface doesn't show up.

 

[oguz]

hi,

but it seems that when accessing the public IP the rancher interface doesn't show up.

what is your network setup like? are you forwarding ports to your containers?

to access from public internet to your container, either your container will need a public IP or you will need to do portforwarding from your PVE to the container to expose the services.

 

[Dendi Suhubdy]

So I have 8 servers that all have public IPs. The containers could also obtain public IPs.

When I tried to access https://<public_ip_container> it doesn't quite work. My browser goes stuck in a loop going to http and then https then back to http.

Anything that I can provide from the LXC Ubuntu 18.04 container so that it can help us here?

 

[oguz]

My browser goes stuck in a loop going to http and then https then back to http.

sounds like a redirect loop? maybe you did something wrong while configuring the web server?

can you try curling your IP with -v flag? (also add -L for following redirect) to see what happens?

 

[r.jochum]

@Dendi Suhubdy what kind of Loadbalancer do you use for Rancher? Can you ssh in one node and launch "kubectl -n cattle-system get service -o wide" ?

I did a production setup like this

 

[Dendi Suhubdy]

@Dendi Suhubdy what kind of Loadbalancer do you use for Rancher? Can you ssh in one node and launch "kubectl -n cattle-system get service -o wide" ?

I did a production setup like this

Is that done all on Proxmox VE cluster?

 

[Dendi Suhubdy]

From my server
root@rancher:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 306af42d482e rancher/rancher "entrypoint.sh" 9 minutes ago Up 1 second 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp suspicious_saha


from my client

╰─ curl -v http://xxx.xxx.xxx.xxx ─╯ * Trying xxx.xxx.xxx.xxx... * TCP_NODELAY set * Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) port 80 (#0) > GET / HTTP/1.1 > Host: xxx.xxx.xxx.xxx > User-Agent: curl/7.64.1 > Accept: */* > * Recv failure: Connection reset by peer * Closing connection 0 curl: (56) Recv failure: Connection reset by peer ╭─░▒▓  ╱  ~ ▓▒░·······················································░▒▓ 56 ✘ ╱ at 10:18:04  ▓▒░─╮ ╰─ curl -v https://xxx.xxx.xxx.xxx ─╯ * Trying xxx.xxx.xxx.xxx... * TCP_NODELAY set * Connection failed * connect to xxx.xxx.xxx.xxx port 443 failed: Connection refused * Failed to connect to xxx.xxx.xxx.xxx port 443: Connection refused * Closing connection 0 curl: (7) Failed to connect to xxx.xxx.xxx.xxx port 443: Connection refused

 

[Dendi Suhubdy]

@Dendi Suhubdy what kind of Loadbalancer do you use for Rancher? Can you ssh in one node and launch "kubectl -n cattle-system get service -o wide" ?

I did a production setup like this

I used the command `sudo docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher` to start rancher. Is that not the right command for proxmox LXC's?

Is there a tutorial where I could follow and install rancher or rancher OS?

Some how I found these tutorials/links
- https://github.com/rancher/os/issues/2938
- https://github.com/rancher/os/pull/2943
- https://github.com/rancher/os/issues/2899

 

[r.jochum]

@Dendi Suhubdy No because of company policy not on Proxmox sadly :/

There's a big difference between rancherOS and rancher k8s|k3s.

What do you want to use?

 

[Dendi Suhubdy]

Either way I'm fine, as long as I can access kubernetes first via the GUI and later via kubectl.

 

[r.jochum]

Then follow my tutorial I linked, you might want to do one thing different. MariaDB Galera on its own VM's.

Also disable the proxmox firewall if things aren't working as expected.

EDIT: the K8S uses ETCD which takes a lot of IOPs, K3S with Galera is HA and takes lots lesser IOPs and is great to use.

 

[Dendi Suhubdy]

Thanks @r.jochum! Will do!

Turns out according to this https://forum.proxmox.com/threads/docker-on-lxc-or-vm.52872/ it's better to not use the LXC for the rancher server. It's better to run the Rancher OS in a VM instead in a LXC.

I guess I will need to figure this out.

 

[r.jochum]

Sorry i missed the "LXC" part :/

 

[Dendi Suhubdy]

No probs, I will post my findings here. I saw https://www.youtube.com/watch?v=_Od8Z3iv54M&t=9s and it seems to work fine

 

[Dendi Suhubdy]

I decided to use a Ubuntu 18.04 VM and then use https://rancher.com/quick-start/ to install. Works like a charm!