Hey Tmanok,
I am not a coder by trade, so things like github are alien to me, I have a textfile, thats it.
The proxmox-offline-mirror tool might not be intended for how I use it, but it does the job pretty well.
Its not eating up much storage at all actually, the diskspace it uses at the moment is about 150Gb.
Besides them using hardlinks to setup the snapshots, which saves space, you can also pick-and-choose the downloaded packages.
When configuring the mirrors using the tool you can set filters for packages you know are not needed on a Proxmox node, like games and libreoffice.You can exclude complete categories, without having to go in much detail.
With regards to having controll over the what snapshot each proxmox node uses when updating, the controll is still there in this case.
Even though I have the nodes pointing towards the symlink to 'latest', they only update when you manually trigger the button.
If I need them to stay on a certain level I can always change them to point to any off the last 7 snapshots available on the offline-mirror.
For me the script was a need-to-have, as otherwise I could just not get consistent results when updating my nodes inside mainland China.
Since the offline-mirror also reduces the window where a node is in maintenance mode, I found that its also a nice-to-have for my cluster in the EU.
I will try and add the script and the proxmox-offline-mirror config file to this post using the code snippet.
Anyone using the script be advised, this was created to scratch my particular itch, it might not fit your needs.
Also know that I am not a code, I started out with simple tests, created a bash script, but in the end I had Copilot validate the script.
Copilot added some validations to the script and simplified it. (so AI was involved)
The configuration file used for the proxmox-offline-mirror tool (it points to a dutch repo, might wanna change that)
Note the skipped packages and package sections.
The config for our production cluster has the enterprise repository in it, the one below points to the no-subscription repo.
Code:
mirror: debian_trixie_security
architectures amd64
architectures all
base-dir /var/lib/proxmox-offline-mirror/mirrors/
ignore-errors false
key-path /usr/share/keyrings/debian-archive-keyring.gpg
repository deb http://security.debian.org/debian-security trixie-security main contrib non-free-firmware
skip-packages "linux-source-*"
skip-packages "*-doc"
skip-packages "thunderbird*"
skip-packages "wine*"
skip-packages "libreoffice*"
skip-sections debug
skip-sections games
skip-sections fonts
skip-sections gnome
skip-sections documentation
skip-sections education
skip-sections electronics
sync true
verify true
mirror: debian_trixie_main
architectures amd64
architectures all
base-dir /var/lib/proxmox-offline-mirror/mirrors/
ignore-errors false
key-path /usr/share/keyrings/debian-archive-keyring.gpg
repository deb http://debian.snt.utwente.nl/debian/ trixie main contrib non-free-firmware
skip-packages "linux-source-*"
skip-packages "*-doc"
skip-packages "thunderbird*"
skip-packages "wine*"
skip-packages "libreoffice*"
skip-sections debug
skip-sections games
skip-sections fonts
skip-sections gnome
skip-sections documentation
skip-sections education
skip-sections electronics
sync true
verify true
mirror: debian_trixie_updates
architectures amd64
architectures all
base-dir /var/lib/proxmox-offline-mirror/mirrors/
ignore-errors false
key-path /usr/share/keyrings/debian-archive-keyring.gpg
repository deb http://debian.snt.utwente.nl/debian/ trixie-updates main contrib non-free-firmware
skip-packages "linux-source-*"
skip-packages "*-doc"
skip-packages "thunderbird*"
skip-packages "wine*"
skip-packages "libreoffice*"
skip-sections debug
skip-sections games
skip-sections fonts
skip-sections gnome
skip-sections documentation
skip-sections education
skip-sections electronics
sync true
verify true
mirror: proxmox-ve-no-subscription
architectures amd64
architectures all
base-dir /var/lib/proxmox-offline-mirror/mirrors
ignore-errors false
key-path /usr/share/keyrings/proxmox-archive-keyring.gpg
repository deb http://download.proxmox.com/debian/pve trixie pve-no-subscription
sync true
verify true
Below is the script itself :
Code:
#!/usr/bin/env bash
set -euo pipefail
log() {
echo "$(date '+%F %T') $*"
}
# Mirror locations
declare -A REPOS=(
[debian_trixie_main]="/var/lib/proxmox-offline-mirror/mirrors/debian_trixie_main"
[debian_trixie_security]="/var/lib/proxmox-offline-mirror/mirrors/debian_trixie_security"
[debian_trixie_updates]="/var/lib/proxmox-offline-mirror/mirrors/debian_trixie_updates"
[proxmox-ve-no-subscription]="/var/lib/proxmox-offline-mirror/mirrors/proxmox-ve-no-subscription"
)
# Number of snapshots to retain
KEEP_SNAPSHOTS=7
# Include hidden entries and avoid literal glob expansion when no matches exist
shopt -s dotglob nullglob
cleanup_snapshots() {
local topdir="$1"
local mirror="$2"
local snapshots=()
local newest
# Collect snapshot directories sorted by modification time (newest first)
mapfile -t snapshots < <(
find "$topdir" \
-mindepth 1 \
-maxdepth 1 \
-type d \
! -name latest \
-printf '%T@ %p\n' |
sort -rn |
cut -d' ' -f2-
)
if (( ${#snapshots[@]} == 0 )); then
log "WARNING: No snapshots found for $mirror"
return 1
fi
newest="${snapshots[0]}"
log "Keeping latest snapshot: $(basename "$newest")"
# Update the "latest" symlink atomically
ln -sfn "$newest" "$topdir/latest"
# Remove snapshots older than the retention count
if (( ${#snapshots[@]} > KEEP_SNAPSHOTS )); then
for oldsnap in "${snapshots[@]:$KEEP_SNAPSHOTS}"; do
log "Removing old snapshot: $(basename "$oldsnap")"
proxmox-offline-mirror mirror snapshot remove \
"$mirror" \
"$(basename "$oldsnap")"
done
fi
}
for mirror in "${!REPOS[@]}"; do
log "Creating snapshot for $mirror"
proxmox-offline-mirror mirror snapshot create "$mirror"
cleanup_snapshots "${REPOS[$mirror]}" "$mirror"
done
log "Running garbage collection"
proxmox-offline-mirror mirror gc
log "Completed successfully"
Feel free to use the script / alter it to your needs, but I can not offer much help if the script does not work for you.
The script was added to our internal repository, as a service, and this service is started (by a timer, not cron) each night at 01:00.
Nginx is running on the internal repo to publish the repositories to the Proxmox nodes.
As an example here is the proxmox.sources apt file :
Code:
Types: deb
URIs: http://<FQDN-of-internal-repo-host>/proxmox-ve-no-subscription/latest
Suites: trixie
Components: pve-no-subscription
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
