Proxmox Built in Secret Manager

[devb0x]

Hi all,

Does Proxmox currently have a way to handle secrets in configuration files or VMs/LXCs?

I am leveraging Terraform to deploy VMs using cloud-init templates.

As part of my configuration, I use additional cloud-init settings to customize and configure virtual machines during deployment.

During deployment, Terraform reads the additional templates from /var/lib/vz/snippets/*.yaml on the Proxmox node it's deploying on and then applies these steps to the VM during the build.

For the most part, these configurations include basic tasks like hardening the VM and installing Docker. However, I’ve encountered scenarios where I need to include secrets in the configuration for more complex deployments.

For obvious reasons, I don’t want to store secrets in plain text. To address this, I’m using HashiCorp's Vault API to fetch secrets during deployment. However, even using short-lived tokens isn't ideal, as it makes the process less repeatable—requiring constant token rotation, which adds complexity and friction to the workflow.

Additionally, this approach still requires me to provide the Vault token in the template, which isn’t secure or efficient.

Does Proxmox currently offer a secret store, or are there any plans to integrate one for managing configurations like this in the future?
It would be great if there were a way to set up secret injection by referencing secrets directly in the VM or template using something like ${my_vault_token}.

 

[gfngfn256]

How about Debian's native secret-tool with keyrings?
From my current Proxmox node:

# apt search secret-tool
Sorting... Done
Full Text Search... Done
libsecret-tools/stable 0.20.5-3 amd64
  tool for storing and retrieving GObject passwords

 

[devb0x]

I looked at the docs but I don't see any way that I could deploy this and not have to put secrets in plain text into the config file that gets deployed. Seems to leave in the same boat as what I am doing now.

#cloud-config
runcmd:
  - apt update
  - apt install -y qemu-guest-agent git jq curl
  - systemctl start qemu-guest-agent
  - systemctl -w vm.max_map_count=262144
  - git clone https://github.com/wuzuh/wuzuh.git /home/debian/
  - chown -R debian:debian /home/debian/wuzuh
  - bash /root/setup_secrets.sh
  - bash /root/install_docker.sh
  - bash /root/setup_secrets.sh
  - su - debian -c "cd /home/debian/wuzuh && docker-compose -f generate-indexer-certs.yml run --rm generator"
  - sleep 30
  - su - debian -c "cd /home/debian/wuzuh && docker-compose up -d"

write_files:
  - path: /etc/ssh/sshd_config
    content: |
      # SSH configuration
      Port 33
      PermitRootLogin no
      PasswordAuthentication no
      ChallengeResponseAuthentication no
      UsePAM yes
      PermitEmptyPasswords no
      AuthenticationMethods publickey
      AllowTcpForwarding yes
      X11Forwarding no
      MaxAuthTries 3
      ClientAliveInterval 300
      ClientAliveCountMax 2
      LogLevel VERBOSE

  - path: /root/install_docker.sh
    permissions: '0755'
    content: |
      #!/bin/bash
      # Install Docker
      curl -fsSL https://get.docker.com -o get-docker.sh
      sudo sh get-docker.sh
      printf '\nDocker installed successfully\n\n'

      printf 'Waiting for Docker to start...\n\n'
      sleep 5

      # Install Docker Compose
      COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep 'tag_name' | cut -d\" -f4)
      sudo curl -L https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-$(uname -s)-$(uname -m) > /usr/local/bin/docker-compose
      sudo chmod +x /usr/local/bin/docker-compose
      printf '\nDocker Compose installed successfully\n\n'
      sudo docker-compose -v

  - path: /root/setup_secrets.sh
    permissions: '0755'
    content: |
      #!/bin/bash

      # Variables
      VAULT_ADDR="https://my.vault.com"
      VAULT_TOKEN="My_Vault_Token"
      VAULT_PATH="op/vaults/automation/items/wuzuh_creds"
      ENV_FILE="/home/debian/wuzuh/.env"

      # Fetch secrets from Vault for specific variables
      INDEXER_USERNAME=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
      $VAULT_ADDR/v1/$VAULT_PATH | jq -r '.data.username')
      INDEXER_PASSWORD=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
      $VAULT_ADDR/v1/$VAULT_PATH | jq -r '.data.password')
      API_USERNAME=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
      $VAULT_ADDR/v1/$VAULT_PATH | jq -r '.data.api_username')
      API_PASSWORD=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
      $VAULT_ADDR/v1/$VAULT_PATH | jq -r '.data.api_password')
      DASHBOARD_USERNAME=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
      $VAULT_ADDR/v1/$VAULT_PATH | jq -r '.data.dashboard_username')
      DASHBOARD_PASSWORD=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
      $VAULT_ADDR/v1/$VAULT_PATH | jq -r '.data.dashboard_password')

      # Validate fetched secrets
      if [[ -z "$INDEXER_USERNAME" || -z "$INDEXER_PASSWORD" || -z "$API_USERNAME" || -z "$API_PASSWORD" || -z "$DASHBOARD_USERNAME" || -z "$DASHBOARD_PASSWORD" ]]; then
      echo "Error: One or more secrets could not be fetched from Vault."
      exit 1
      fi

      # Update .env file with the fetched secrets
      if [ -f "$ENV_FILE" ]; then
      echo "Updating $ENV_FILE with fetched secrets..."
      sed -i "s|^INDEXER_USERNAME=.*|INDEXER_USERNAME=$INDEXER_USERNAME|" "$ENV_FILE"
      sed -i "s|^INDEXER_PASSWORD=.*|INDEXER_PASSWORD=$INDEXER_PASSWORD|" "$ENV_FILE"
      sed -i "s|^API_USERNAME=.*|API_USERNAME=$API_USERNAME|" "$ENV_FILE"
      sed -i "s|^API_PASSWORD=.*|API_PASSWORD=$API_PASSWORD|" "$ENV_FILE"
      sed -i "s|^DASHBOARD_USERNAME=.*|DASHBOARD_USERNAME=$DASHBOARD_USERNAME|" "$ENV_FILE"
      sed -i "s|^DASHBOARD_PASSWORD=.*|DASHBOARD_PASSWORD=$DASHBOARD_PASSWORD|" "$ENV_FILE"
      else
      echo "Error: .env file not found at $ENV_FILE."
      exit 1
      fi

      echo ".env file updated successfully!"

 

[LnxBil]

I never understood the "locality" of cloudinit. What about using another automation toolchain like ansible to setup the VMs "from the outside"?

 

[devb0x]

I never understood the "locality" of cloudinit. What about using another automation toolchain like ansible to setup the VMs "from the outside"?

I could use ansible but that would either be yet another manual process or it would be yet another thing to maintain if you use semaphore or tower.
I am not against anisble. It just kinda defeats the point of using terraform to handle everything. When i say everything I really mean everything. Anything from HA state, storage, to network gets handled by terraform. In my setup terraform runs in a github pipeline that reaches out to proxmox. Cloud init gets triggered by terraforms build process and sets up my VM the way I want right from the beginning. I can install everything I want, harden the VM, etc. all from the template. It's much nicer to just push a single commit to github rather then manually go in and have to take a buch of other steps to add additional configuration.

 

[LnxBil]

It's much nicer to just push a single commit to github rather then manually go in and have to take a buch of other steps to add additional configuration.

You could have that with ansible too, yet I see that you have to change everything, which is often a hard pass.

What we did in the past was to have a server for secret data that provide it based on the querying IP address, so that you need to spoof the IP to get to the data. The url was the same for every client, yet it got only the data relevant for the querying client. Maybe such a setup would be applicable to your secrets problem?