[SOLVED] automating with bpg/proxmox; How to find URL and checksum of LXC images?

[steevestroke]

Howdy, folks.

I'm trying to populate my proxmox servers' LXC templates with the bpg/proxmox Terraform provider. Pulling from Debian's site, that would look something like this:

resource "proxmox_virtual_environment_download_file" "debian-12-amd64-lxc" {
  content_type = "vztmpl"
  datastore_id = "local"
  node_name = "<redacted>"
  url = "https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.tar.xz"
  checksum = "336129df1c96765434d7725bc9651ad929c119480d14da14a2c1b892ba31a823700739344656fc42cab12af19ffccb7ed5f98339c58d163ebefa4cf40e70e319"
  checksum_algorithm = "sha512"
  upload_timeout = 300
}

However I would like to pull the images/templates as they would be pulled from the GUI console or the pveam tool. So in this case:

$sudo pveam available | grep -e 'debian.*12.*standard'
system          debian-12-standard_12.2-1_amd64.tar.zst

The problem is, for the code, I need the URL and checksum of the Debian 12 standard image. I've looked everywhere in the GUI and in the CLI and I cannot find those values.

Any advice?

 

[fabian]

/var/lib/pve-manager/apl-info should contain the information you are looking for

 

[steevestroke]

/var/lib/pve-manager/apl-info should contain the information you are looking for

Thanks for that pointer. We are getting somewhere. There are a couple of files in this directory:

$ ls -la
total 80
drwxr-xr-x 2 root root  4096 Jan 24 01:18 .
drwxr-xr-x 4 root root  4096 Jan 24 10:30 ..
-rw-r--r-- 1 root root 10654 Jan 24 01:18 download.proxmox.com
-rw-r--r-- 1 root root 57593 Jan 24 01:18 releases.turnkeylinux.org

The Turnkey Linux index has a Location: field with a full URL:

$ grep -A4 -B4 'debian.*12' releases.turnkeylinux.org
...
Package: turnkey-bagisto
Version: 18.0-1
Type: lxc
OS: debian-12
Section: turnkeylinux
Architecture: amd64
Location: http://mirror.turnkeylinux.org/turnkeylinux/images/proxmox/debian-12-turnkey-bagisto_18.0-1_amd64.tar.gz
Infopage: http://www.turnkeylinux.org/bagisto
ManageUrl: http://__IPADDRESS__/
...

While the Proxmox one appears to have a shortened URL prefix of "system":

$ grep -A4 -B4 'debian.*12.*standard' download.proxmox.com
Infopage: https://pve.proxmox.com/wiki/Linux_Container#pct_supported_distributions
Description: Debian 11 Bullseye (standard)
 A small Debian Bullseye system including all standard packages.

Package: debian-12-standard
Version: 12.2-1
Type: lxc
OS: debian-12
Section: system
Maintainer: Proxmox Support Team <support@proxmox.com>
Architecture: amd64
Location: system/debian-12-standard_12.2-1_amd64.tar.zst
md5sum: 0c40b2b49499c827bbf7db2d7a3efadc
sha512sum: 1846c5e64253256832c6f7b8780c5cb241abada3ab0913940b831bf8f7f869220277f5551f0abeb796852e448c178be22bd44eb1af8c0be3d5a13decf943398a
Infopage: https://pve.proxmox.com/wiki/Linux_Container#pct_supported_distributions
Description: Debian 12 Bookworm (standard)

The Terraform provider is not able to expand the "system" prefix to the correct URL so I'll need to find that value. Perhaps someone here knows? I may try to determine it from looking at router traffic.

This might also be an interesting PR against the Terraform provider... Meaning allowing "system" prefix and expanding it.

 

[steevestroke]

A couple of interesting observations here for posterity. Here's some example code:

resource "proxmox_virtual_environment_download_file" "debian-12-amd64-lxc" {
  content_type = "vztmpl"
  datastore_id = "local"
  node_name = "<redacted>"
  url = "https://na.cdn.proxmox.com/images/system/debian-12-standard_12.2-1_amd64.tar.zst"
  checksum = "1846c5e64253256832c6f7b8780c5cb241abada3ab0913940b831bf8f7f869220277f5551f0abeb796852e448c178be22bd44eb1af8c0be3d5a13decf943398a"
  checksum_algorithm = "sha512"
  upload_timeout = 300
}

The URL prefix I was able to divine from other forum posts -- I looked at the pve-manager code but that didn't lead anywhere -- was:

https://download.proxmox.com/images/system/

That URL throws an error in the TF Provider due to the TLS certificate not having download.proxmox as a CN nor as a SAN. So I adjusted to "na.cdn...." and that got rid of the TLS error. I assume there's some CNAME magic going on there.

I then used the divined prefix and added the filename and checksum from the index file in /var however I'm getting a 404. That appears to be the current Debian 12 Standard image/name so I must not yet have the correct prefix.

 

[steevestroke]

Got it!

# pveam download local debian-12-standard_12.2-1_amd64.tar.zst
downloading http://download.proxmox.com/images/system/debian-12-standard_12.2-1_amd64.tar.zst to /var/lib/vz/template/cache/debian-12-standard_12.2-1_amd64.tar.zst
calculating checksum of existing file...OK, got correct file already, no need to download

resource "proxmox_virtual_environment_download_file" "debian-12-amd64-lxc" {
  content_type = "vztmpl"
  datastore_id = "local"
  node_name = "<redacted>"
  url = "http://download.proxmox.com/images/system/debian-12-standard_12.2-1_amd64.tar.zst"
  checksum = "1846c5e64253256832c6f7b8780c5cb241abada3ab0913940b831bf8f7f869220277f5551f0abeb796852e448c178be22bd44eb1af8c0be3d5a13decf943398a"
  checksum_algorithm = "sha512"
  upload_timeout = 300
}

No TLS gets around the cert issue but mainly its just that the file is available via http (and not at all via https?). No TLS is fine in this case because we have a checksum.

 

[fabian]

it's also advisable to use download.proxmox.com since the geo-distributed hosts behind that might change at any point, we don't hide them but we also don't recommend using them directly since we might re-organize/shutdown for maintainance/..

if you want to go the extra mile - the index files are also available online (obviously, since pveam needs to fetch updated versions of them ) and are signed with GPG