pve-ssl.pem replaced, but its old version is restored after ~1 day automatically?

[divStar]

Hello everyone,

I am having an issue with the pve-ssl.pem file on my standalone Proxmox node.

I have a created a terraform script, that creates a custom domain certificate, which contains one more domain so I can access the Proxmox web UI not just via IP, the host or the extended host, but yet another host. I signed the certificate using the Proxmox CA. Both the Proxmox node and my client machine (MacBook) trust the Proxmox CA.

Instead of staying overwritten though, the pve-ssl.pem file is automatically restored after ~1 day. However: neither did I run tofu destroy to undeploy the changes nor did I set up any kind of cron-job or run the pvecm updatecerts command as I didn't intend to restore the file. Besides: only the pve-ssl.pem file seems to be about 1 day newer than the pve-ssl.key file:

root@sanctum:/etc/pve/nodes/sanctum# ll
...
-rw-r----- 1 root www-data 1675 Apr 16 02:38 pve-ssl.key
-rw-r----- 1 root www-data 1797 Apr 17 04:45 pve-ssl.pem

Here is the Terraform script, that creates the new pve-ssl.key and pve-ssl.pem and replaces the new one: https://github.com/divStar/homelab/tree/master/modules/host-setup/modules/update-ssl.

Spoiler: See main.tf

/**
 * # Update `pve-ssl` certificate with additional domain(s).
 *
 * Handles fetching the Proxmox CA certificate and key,
 * generating the `pve-ssl` certificate with additional
 * domain(s) and IP(s) anew and copying of it back onto
 * the host.
 */
locals {
  timestamp = formatdate("YYYYMMDD", time_static.backup_timestamp.rfc3339)
}

resource "time_static" "backup_timestamp" {}

# Fetch Proxmox CA public certificate
resource "ssh_resource" "proxmox_ca_cert" {
  host        = var.ssh.host
  user        = var.ssh.user
  private_key = file(var.ssh.id_file)

  # when = "create"

  commands = [
    "cat ${var.proxmox_root_ca.pve_root_cert}"
  ]
}

# Fetch Proxmox CA key
resource "ssh_resource" "proxmox_ca_key" {
  host        = var.ssh.host
  user        = var.ssh.user
  private_key = file(var.ssh.id_file)

  # when = "create"

  commands = [
    "cat ${var.proxmox_root_ca.pve_root_key}"
  ]
}

# Generate private key for the SSL certificate
resource "tls_private_key" "pve_ssl_key" {
  algorithm = var.proxmox_domain_cert.private_key.algorithm
  rsa_bits  = var.proxmox_domain_cert.private_key.rsa_bits
}

# Create certificate request
resource "tls_cert_request" "pve_ssl_cert_request" {
  private_key_pem = tls_private_key.pve_ssl_key.private_key_pem

  subject {
    common_name         = var.proxmox_domain_cert.subject.common_name
    organization        = var.proxmox_domain_cert.subject.organization
    organizational_unit = var.proxmox_domain_cert.subject.organizational_unit
    country             = var.proxmox_domain_cert.subject.country
    locality            = var.proxmox_domain_cert.subject.locality
    province            = var.proxmox_domain_cert.subject.province
  }

  dns_names    = var.proxmox_domain_cert.dns_names
  ip_addresses = var.proxmox_domain_cert.ip_addresses
}

# Sign the certificate with the CA
resource "tls_locally_signed_cert" "pve_ssl_cert" {
  cert_request_pem   = tls_cert_request.pve_ssl_cert_request.cert_request_pem
  ca_cert_pem        = ssh_resource.proxmox_ca_cert.result
  ca_private_key_pem = ssh_resource.proxmox_ca_key.result

  validity_period_hours = var.proxmox_domain_cert.validity_period_hours

  allowed_uses = [
    "key_encipherment",
    "digital_signature",
    "server_auth",
  ]
}

# Back up existing certificates
resource "ssh_resource" "backup_existing_certs" {
  host        = var.ssh.host
  user        = var.ssh.user
  private_key = file(var.ssh.id_file)

  # when = "create"

  commands = [
    "cp /etc/pve/nodes/${var.proxmox_host}/pve-ssl.pem /etc/pve/nodes/${var.proxmox_host}/pve-ssl.pem.backup.${local.timestamp} || true",
    "cp /etc/pve/nodes/${var.proxmox_host}/pve-ssl.key /etc/pve/nodes/${var.proxmox_host}/pve-ssl.key.backup.${local.timestamp} || true"
  ]
}

# Install the new certificate and key on the Proxmox server
resource "ssh_resource" "install_pve_cert" {
  depends_on = [ssh_resource.backup_existing_certs]

  host        = var.ssh.host
  user        = var.ssh.user
  private_key = file(var.ssh.id_file)

  # when = "create"

  # Install the private key
  file {
    content     = tls_private_key.pve_ssl_key.private_key_pem
    destination = "/etc/pve/nodes/${var.proxmox_host}/pve-ssl.key"
    permissions = "0640"
  }

  # Install the certificate
  file {
    content     = tls_locally_signed_cert.pve_ssl_cert.cert_pem
    destination = "/etc/pve/nodes/${var.proxmox_host}/pve-ssl.pem"
    permissions = "0640"
  }

  # Restart services to apply the new certificate
  commands = [
    "systemctl restart pveproxy"
  ]
}

When I run this Terraform script, it replaces the certificate files and after the pveproxy restart I have no issues accessing Proxmox from the additional host, that I added.

Is there some sort of cron-job or the likes, that restores the certificate?
If so: is there a way to use a custom SSL certificate for the GUI / API?

Should I perhaps just add the certificate as a new one and somehow point the pveprox configuration to use it instead?