Proxmox behind HAProxy SSL backend and frontend results in no encryption at all.

derFelix

New Member
Sep 23, 2024
5
0
1
Hello,
I wanted to use HAProxy in front of Proxmox to make Proxmox accessible from the internet for administration by other people.
I have set up a frontend bound to Port 443 with my wildcard cert aquired with certbot.
I have been experimenting for hours to find a configuration for the backend which would work without results.
For some unknown reason my browser complains about no (!) encryption at all and thus won't connect.

I have put my whole HAProxy configuration in a pastebin: https://pastebin.com/sBTqtc8m
What I tried so far:
  • Alle the uncommented backend server configurations
  • HTTP to HTTPS redirection (didn't work in general)
  • Proxmox node domain and acme configuration -> proxmox.mydomain.dev
I wasn't able to find anything helpful on the internet that worked for me.
 
Hello, @derFelix

1) Have you verified the config? Probably with
haproxy -f /etc/haproxy/haproxy.cfg -c

2) What do haproxy logs say?

3) If there are no sufficient hints in the logs, I would start with very basic configuration.
That is, without frontend www-http (so no bind *:80 at all),
without http-request set-header , http-request add-header, http-response set-header directives.
In the backend section: mode http not mode tcp . No additional option directives.

P.S. Probably unrelated, but I wonder why you put acl letsencrypt-acl in a https frontend instead of in a http frontend. AFAIR, Letsencrypt challenge requires http access to avoid "chicken or the egg" problem.
 
  • Like
Reactions: Johannes S
Probably unrelated, but I wonder why you put acl letsencrypt-acl in a https frontend instead of in a http frontend. AFAIR, Letsencrypt challenge requires http access to avoid "chicken or the egg" problem.
Exactly, the ACME protocol needs http access. An alternative is the usage of the DNS challenge and this is what I do. Domains are available for under ten bucks a year and it makes things a lot easier to be able to use something like homelab.yourdomain.de for stuff like this.
 
I get that http makes more sense for letsencrypt buuuut the guide from digitalocean I found on the certbot site actually puts it in https. All I can say is that it works like that, I have cut out about 9 services from the pastebin I successfully use over https (with backends running on http and https). Renewal's also no problem.

Now back to the actual topic:
I actually never used tha validation feature because I've checked journalctl many times which gives the same output.
The validation output contains no lines regarding pve backend or frontends.

After removing mode tcp which I forgot beforehand I saw this new Error:
Code:
jun 18 20:36:23 HAProxy-LB-Master haproxy[23450]: 192.168.180.105:57642 [18/Jun/2026:20:36:23.990] www-https~ pve/pve 0/0/0/4/4 301 248 - - ---- 1/1/0/0/0 0/0 "GET https://proxmox.mydomain.dev/ HTTP/2.0"
The redirection is obviously not working.

With ssl verify none I got 503:
Code:
Jun 18 20:40:02 HAProxy-LB-Master haproxy[23610]: 192.168.180.105:57720 [18/Jun/2026:20:40:02.811] www-https~ pve/pve 0/0/-1/-1/28 503 0 - - sC-- 1/1/0/0/3 0/0 "GET https://proxmox.mydomain.dev/ HTTP/2.0"

After removing the set/added headers from the fromtend with http-request del-header no change.
Removing them directly from the frontend including HSTS reuslted in no change.
Removing all options also changed nothing.

Same result when using the other two server options with ssl.

What makes me really frustrated is that portainer works flawlessly with ssl.

Edit: The 301 is actually a redirect loop to itself. Proxmox probably tries to redirect to https.
 
Last edited:
Two things in your config I don't get_

Your PVE backend explicitly disables ssl doesn't it?
Code:
        server pve 192.168.0.10:8006
        #server pve 192.168.0.10:8006 ssl verify required ca-file /etc/haproxy/ssl/mydomain.dev/cert.pem
        #server pve 192.168.0.10:8006 ssl sni hdr(host) verify none
        #server pve 192.168.0.10:8006 ssl verify none

If don't miss anything this backend won't use ssl for any connection to the backends. So even if the frontend will still be served via https the backend won't follow. This is my guess at least I'm not an haproxy expert and might be wrong. Did you try another reverse proxy like caddy or traefik who have acme already included? Even if you prefer to use haproxy this would allow to check whether that the error is in the haproxy configuration instead of some other factor.
 
@derFelix

I would say that
server pve 192.168.0.10:8006 ssl verify none
should work.
If I understand your posts, you have already tried that without success. But maybe not with the other directives modified according to our suggestions?...

https://www.haproxy.com/documentation/haproxy-configuration-manual/latest/#8.5
describes termination_field.

sC in sC-- means:

s : the server-side timeout expired while waiting for the server to send or receive data.
C : the proxy was waiting for the CONNECTION to establish on the server. The server might at most have noticed a connection attempt.


Have you checked that from the proxy server itself you are able to connect to 192.168.0.10:8006?
For instance with wget or curl?

Will you get some better result or better hint if you remove from the config the line
errorloc 503 https://somethinghappened503.mydomain.dev
?
 
  • Like
Reactions: Johannes S
If HAProxy is terminating SSL on the frontend and then forwarding plain HTTP to the backend, the traffic between HAProxy and Proxmox won't be encrypted. That's expected unless SSL is also enabled on the backend connection.

I'd double check the HAProxy backend configuration and verify whether ssl is being used on the server line. It can also help to test with curl -vk or inspect the connection directly to confirm where TLS is being terminated.

One thing that often causes confusion is that the browser still shows HTTPS because the client to HAProxy connection is encrypted, even if the backend traffic is not. Have you checked the HAProxy logs or run a packet capture between HAProxy and Proxmox to verify what's actually happening?
 
  • Like
Reactions: Johannes S
You are using an http mode frontend with a tcp mode backend for pve, which doesn't make much sense: haproxy will terminate the ssl tunnel and send clear text traffic to PVE which won't work. Change the backend mode to http and use something like this for the backend server:

Code:
backend pve
        mode http
        server pve 192.168.0.10:8006 ssl verify none check inter 10s
 
  • Like
Reactions: Johannes S