[TUTORIAL] PVE proxy with kubernetes nginx ingress

r.jochum

Renowned Member
Mar 26, 2018
289
47
68
38
Austria
rene.jochum.dev
Today I moved my Proxmox behind an Kubernetes NGINX Ingress, I did that because I want a CORS, a Let's Encrypt Cert with a single Public IP and port 443.

The Service YAML

Code:
apiVersion: v1
kind: Service
metadata:
  annotations:
    field.cattle.io/ipAddresses: '["10.167.160.10"]'
  name: pve01
  namespace: my-pve-namespace
spec:
  clusterIP: None
  ports:
  - name: pve
    port: 8006
    protocol: TCP
    targetPort: 8006
  sessionAffinity: None
  type: ClusterIP


And the ingress yaml

Code:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    ingress.kubernetes.io/configuration-snippet: "proxy_set_header Host $http_host;\nproxy_set_header
      X-Real-IP $remote_addr;\nproxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      \     \nproxy_http_version 1.1;\nproxy_set_header Upgrade $http_upgrade;\nproxy_set_header
      Connection $connection_upgrade;      "
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/cors-allow-headers: Authorization
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/server-snippet: |-
      client_max_body_size 100G;
      server_tokens off;
      proxy_hide_header X-Powered-By;
  generation: 2
  managedFields:
  name: pve01
  namespace: my-pve-namespace
spec:
  rules:
  - host: pve01.mydomain.com
    http:
      paths:
      - backend:
          serviceName: pve01
          servicePort: 8006
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - pve01.mydomain.com
    secretName: pve01-mycomain-com-tls
 
Last edited:
It works for me with a little modification:

YAML:
---

apiVersion: v1
kind: Service
metadata:
  name: pve01
  namespace: networking
  annotations:
    field.cattle.io/ipAddresses: '["192.168.10.100"]'
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: pve-port
      port: 8006
      protocol: TCP
      targetPort: 8006
  sessionAffinity: None
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pve01
  namespace: networking
  annotations:
    ingress.kubernetes.io/configuration-snippet: "proxy_set_header Host $http_host;\nproxy_set_header
      X-Real-IP $remote_addr;\nproxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      \     \nproxy_http_version 1.1;\nproxy_set_header Upgrade $http_upgrade;\nproxy_set_header
      Connection $connection_upgrade;      "
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/cors-allow-headers: Authorization
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/server-snippet: |-
      client_max_body_size 100G;
      server_tokens off;
      proxy_hide_header X-Powered-By;
  generation: 2
  managedFields:
spec:
  rules:
    - host: &host pve01.domain.local
      http:
        paths:
          - backend:
              service:
                name: pve01
                port:
                  number: 8006
            path: /
            pathType: ImplementationSpecific
  tls:
    - hosts:
        - *host
      secretName: *host
---

k3s: 1.25.11
nginx ingress controller: 4.7.1
cert-manager: 1.12.2

Thank you!
 
Last edited:
Updated 01-15-2024

YAML:
---

apiVersion: v1
kind: Service
metadata:
  name: pve01
  namespace: networking
  annotations:
    field.cattle.io/ipAddresses: '["192.168.10.100"]'
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: pve-port
      port: 8006
      protocol: TCP
      targetPort: 8006
  sessionAffinity: None
---
apiVersion: v1
kind: Endpoints
metadata:
  name: pve01
subsets:
  - addresses:
      - ip: "192.168.1.100"
    ports:
      - name: https
        port: 8006
        protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pve01
  namespace: networking
  annotations:
    ingress.kubernetes.io/configuration-snippet: "proxy_set_header Host $http_host;\nproxy_set_header
      X-Real-IP $remote_addr;\nproxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      \     \nproxy_http_version 1.1;\nproxy_set_header Upgrade $http_upgrade;\nproxy_set_header
      Connection $connection_upgrade;      "
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/cors-allow-headers: Authorization
    nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/server-snippet: |-
      client_max_body_size 100G;
      server_tokens off;
      proxy_hide_header X-Powered-By;
  generation: 2
  managedFields:
spec:
  rules:
    - host: &host pve01.domain.local
      http:
        paths:
          - backend:
              service:
                name: pve01
                port:
                  number: 8006
            path: /
            pathType: ImplementationSpecific
  tls:
    - hosts:
        - *host
      secretName: *host
---

k3s: 1.28.5
nginx ingress controller: 4.9.0
cert-manager: 1.13.3
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!