I figured out the issue on this. It's not an nginx or SSL issue it's a pveproxy issue. I have my pveproxy secured with specific IPs like so:
ALLOW_FROM="192.168.0.1,192.168.0.3,192.168.0.1,192.168.0.5"
DENY_FROM="all"
POLICY="allow"
Obviously not using my real IPs, but I have all the IPs of the...