Proxmox Offline Mirror released!

BloodyIron said:
For the sake of curiosity, what kind of log insights would be useful for you? Jumping in half-way here so sorry if I'm not up to speed on full context :p
Click to expand...
I just meant a kind of "debug logging" feature. In my case it would help me find out at which step exactly POM fails. and which type of connect command was used to connect to the website.
 
  • Like
Reactions: BloodyIron
the connect is not in our code, but in one our dependencies, like I said above ;)
 
user828 said:
Do you need multiple POM subscriptions if you have multiple air-gapped hosts?
Click to expand...
Only if they need to be serviced by different POM instances, e.g. due to the servers being in different physical locations so that using the same POM wouldn't be feasible without breaking the air gap.
 
  • Like
Reactions: user828
fabian said:
I'd use a medium in that case - it ensures the syncing happens correctly now, and also for future changes to POM.
Click to expand...
I'm following up on this question as I feel there's a step missing or I'm misunderstanding something.

  1. I'm running POM on the a system connected to the internet, syncing a number of repositories.
  2. I then use POM to create an offline media, which is an external hard disk.
  3. I then transfer that external hard disk to my air-gapped system.
  4. At this point I need to copy the offline media from the hard disk to a locally available NFS location.
  5. Then update the symlinks to point to the latest snapshot on the NFS share, so pve looks at the latest
For several reasons, it wouldn't be appropriate to have the clients access the external hard disk directly, but it doesn't seem like step 4 above is covered by POM. Is there a POM specific command to transfer the media to another location?

Or should I just use rsync - (rsync -rHptDv?)
 
yeah, the expectation is that you then use the media directly (you could export it via NFS, but of course that would mean interruptions when you re-sync the medium).

syncing the full contents via rsync preserving hardlinks should work.
 
Hi,

First of all many thanks for Proxmox-Offline-Mirror, very useful !!

I was running into the need for a feature for the "current" snapshot link (https://bugzilla.proxmox.com/show_bug.cgi?id=4379) but in the end that was easy to script around.

However I'm also running now into https://bugzilla.proxmox.com/show_bug.cgi?id=4913 , the issue that "/dist/" is always integrated into the url when searching the "Release" and "InRelease" files (https://git.proxmox.com/?p=proxmox-...a17b633ce33dba47594;hb=refs/heads/master#l103).
I haven't managed to work around that one.

Is there any chance on that one getting resolved soon ?
Anything I can do to help ?

Thanks in advance !
 
I haven't started really working on that.. are you also running into it with the jitsi repo, or a different one?
 
In my case it's the kubernetes repo.

I setup the repo with:
Code: 
proxmox-offline-mirror config mirror add \
--id kubernetes_1_31 \
--architectures amd64 \
--repository 'deb https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /' \
--key-path /etc/apt/keyrings/kubernetes-apt-keyring.gpg \
--sync true \
--verify true \
--base-dir /data/mirrors/kubernetes_1_31

Execution gives:
Code: 
# proxmox-offline-mirror mirror snapshot create kubernetes_1_31
Fetching Release/Release.gpg files
-> GET 'https://pkgs.k8s.io/core:/stable:/v1.31/deb//dists///Release.gpg'..
Release.gpg fetch failure: https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.31/deb/dists/Release.gpg: status code 403 (redirected from https://pkgs.k8s.io/core:/stable:/v1.31/deb//dists///Release.gpg)
Fetching InRelease file
-> GET 'https://pkgs.k8s.io/core:/stable:/v1.31/deb//dists///InRelease'..
InRelease fetch failure: https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.31/deb/dists/InRelease: status code 403 (redirected from https://pkgs.k8s.io/core:/stable:/v1.31/deb//dists///InRelease)
Error: Neither Release(.gpg) nor InRelease available!

Unfortunately the kubernetes repo doesn't support file browsing (https://github.com/kubernetes/release/issues/3317) but they point to a opensuse-hosted mirror. You can see that the InRelease files and Release files there are in https://download.opensuse.org/repositories/isv:/kubernetes:/core:/stable:/v1.31/deb/ .

so from there tried with
Code: 
wget 'https://pkgs.k8s.io/core:/stable:/v1.31/deb///Release.gpg
wget https://pkgs.k8s.io/core:/stable:/v1.31/deb///InRelease
(removing the "/dist/") and those download OK.
 
Last edited:
would you mind adding that information to the bug entry? thanks!
 
Hi,

I've installed POM, all repositories are OK but I can't update with Proxmox VE repository : I always have the ""no proxmox ve repository is enabled, you do not get any updates" message when I disable the "official" no-subscription repository and enable the local one.

Here is my sources.list (nothing in sources.list.d) :

Code: 
# deb http://deb.debian.org/debian bookworm main contrib
# deb http://deb.debian.org/debian bookworm-updates main contrib
# deb http://deb.debian.org/debian-security bookworm-security main contrib

deb http://repository.network.local/debian_bookworm bookworm main contrib non-free non-free-firmware
deb http://repository.network.local/debian_bookworm_updates bookworm-updates main contrib non-free non-free-firmware
deb http://repository.network.local/debian_bookworm_security bookworm-security main contrib non-free non-free-firmware

# deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

deb http://repository.network.local/pve_bookworm bookworm pve-no-subscription

And the result of apt update :

Code: 
root@pve-02:~# apt update
Hit:1 http://repository.network.local/debian_bookworm bookworm InRelease
Hit:2 http://repository.network.local/debian_bookworm_updates bookworm-updates InRelease
Hit:3 http://repository.network.local/debian_bookworm_security bookworm-security InRelease
Hit:4 http://repository.network.local/pve_bookworm bookworm InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.

How can I synchronize server with PVE updates without using the official Internet repository ?

Thanks !
 
that message can only check the official, public repositories - it cannot know whether/that your local mirror is a copy of those. as long as the POM snapshots work, there shouldn't be any issue.
 
OK, thanks ! So I suppose my problem is elsewhere, because my repo can't upgrade any packages:

With Internet repository:

Code: 
root@pve-02:~# apt list --upgradable
Listing... Done
ifupdown2/stable 3.2.0-1+pmx11 all [upgradable from: 3.2.0-1+pmx9]
libjs-extjs/stable 7.0.0-5 all [upgradable from: 7.0.0-4]
libpve-access-control/stable 8.2.0 all [upgradable from: 8.1.4]
libpve-cluster-api-perl/stable 8.0.10 all [upgradable from: 8.0.8]
libpve-cluster-perl/stable 8.0.10 all [upgradable from: 8.0.8]
libpve-common-perl/stable 8.2.9 all [upgradable from: 8.2.5]
libpve-guest-common-perl/stable 5.1.6 all [upgradable from: 5.1.4]
libpve-network-perl/stable 0.10.0 all [upgradable from: 0.9.8]
libpve-notify-perl/stable 8.0.10 all [upgradable from: 8.0.8]
libpve-rs-perl/stable 0.9.0 amd64 [upgradable from: 0.8.10]
libpve-storage-perl/stable 8.2.9 all [upgradable from: 8.2.5]
novnc-pve/stable 1.5.0-1 all [upgradable from: 1.4.0-4]
proxmox-archive-keyring/stable 3.1 all [upgradable from: 3.0]
proxmox-backup-client/stable 3.2.9-1 amd64 [upgradable from: 3.2.8-1]
proxmox-backup-file-restore/stable 3.2.9-1 amd64 [upgradable from: 3.2.8-1]
proxmox-firewall/stable 0.6.0 amd64 [upgradable from: 0.5.0]
proxmox-mail-forward/stable 0.3.1 amd64 [upgradable from: 0.2.3]
proxmox-widget-toolkit/stable 4.3.1 all [upgradable from: 4.2.4]
pve-cluster/stable 8.0.10 amd64 [upgradable from: 8.0.8]
pve-container/stable 5.2.2 all [upgradable from: 5.2.0]
pve-docs/stable 8.2.5 all [upgradable from: 8.2.3]
pve-firewall/stable 5.1.0 amd64 [upgradable from: 5.0.7]
pve-ha-manager/stable 4.0.6 amd64 [upgradable from: 4.0.5]
pve-i18n/stable 3.3.0 all [upgradable from: 3.2.4]
pve-manager/stable 8.2.10 amd64 [upgradable from: 8.2.7]
pve-qemu-kvm/stable 9.0.2-4 amd64 [upgradable from: 9.0.2-3]
qemu-server/stable 8.2.7 amd64 [upgradable from: 8.2.4]

And with local one:

Code: 
root@pve-02:~# apt list --upgradable
Listing... Done

But on the local one, I clearly see qemu-server 8.2.7 for example. Apt update result is good, but no updates... I don't know why :/

Thanks.
 
what does "apt list -a qemu-server" output when you only have your mirror configured?
 
and could you post your POM config and APT sources.list entries as well?
 
Code: 
root@pve-02:~# apt list -a qemu-server
Listing... Done
qemu-server/stable,now 8.2.4 amd64 [installed]
qemu-server/stable 8.2.3 amd64
qemu-server/stable 8.2.2 amd64
qemu-server/stable 8.2.1 amd64
qemu-server/stable 8.1.4 amd64
qemu-server/stable 8.1.3 amd64
qemu-server/stable 8.1.2 amd64
qemu-server/stable 8.1.1 amd64
qemu-server/stable 8.1.0 amd64
qemu-server/stable 8.0.10 amd64
qemu-server/stable 8.0.8 amd64
qemu-server/stable 8.0.7 amd64
qemu-server/stable 8.0.6 amd64
qemu-server/stable 8.0.4 amd64
qemu-server/stable 8.0.3 amd64

And conf file for POM :

Code: 
mirror: pve_bookworm
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
        repository deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
        sync false
        verify false

mirror: pve_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
        repository deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
        sync false
        verify false

mirror: pbs_bookworm
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
        repository deb http://download.proxmox.com/debian/pbs bookworm pbs-no-subscription
        sync false
        verify false

mirror: debian_bookworm
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /usr/share/keyrings/debian-archive-bookworm-stable.gpg
        repository deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
        skip-sections debug
        skip-sections games
        sync false
        verify false

mirror: debian_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /usr/share/keyrings/debian-archive-bullseye-automatic.gpg
        repository deb http://deb.debian.org/debian bullseye main contrib non-free
        skip-sections debug
        skip-sections games
        sync false
        verify false

mirror: debian_bookworm_updates
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /usr/share/keyrings/debian-archive-bookworm-automatic.gpg
        repository deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware
        skip-sections debug
        skip-sections games
        sync false
        verify false

mirror: debian_bookworm_security
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /usr/share/keyrings/debian-archive-bookworm-security-automatic.gpg
        repository deb http://deb.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware
        skip-sections debug
        skip-sections games
        sync false
        verify false

mirror: debian_bullseye_security
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /usr/share/keyrings/debian-archive-bullseye-security-automatic.gpg
        repository deb http://deb.debian.org/debian-security bullseye-security main contrib non-free
        skip-sections debug
        skip-sections games
        sync false
        verify false

mirror: pbs_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
        repository deb http://download.proxmox.com/debian/pbs bullseye pbs-no-subscription
        sync false
        verify false

mirror: pmg_bookworm
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
        repository deb http://download.proxmox.com/debian/pmg bookworm pmg-no-subscription
        sync false
        verify false

mirror: ceph_quincy_bookworm
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
        repository deb http://download.proxmox.com/debian/ceph-quincy bookworm no-subscription
        sync false
        verify false

mirror: ceph_reef_bookworm
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
        repository deb http://download.proxmox.com/debian/ceph-reef bookworm no-subscription
        sync false
        verify false

mirror: ceph_octopus_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
        repository deb http://download.proxmox.com/debian/ceph-octopus bullseye main test
        sync false
        verify false

mirror: ceph_pacific_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
        repository deb http://download.proxmox.com/debian/ceph-pacific bullseye main test
        sync false
        verify false

mirror: ceph_quincy_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
        repository deb http://download.proxmox.com/debian/ceph-quincy bullseye main test
        sync false
        verify false

mirror: pmg_bullseye
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
        repository deb http://download.proxmox.com/debian/pmg bullseye pmg-no-subscription
        sync false
        verify false

mirror: debian_bullseye_updates
        architectures amd64
        architectures all
        base-dir /var/lib/proxmox-offline-mirror/mirrors/
        ignore-errors false
        key-path /usr/share/keyrings/debian-archive-bullseye-automatic.gpg
        repository deb http://deb.debian.org/debian bullseye-updates main contrib non-free
        skip-sections debug
        skip-sections games
        sync false
        verify false

medium: nas
        mirrors pbs_bookworm
        mirrors ceph_quincy_bookworm
        mirrors ceph_octopus_bullseye
        mirrors pmg_bookworm
        mirrors pbs_bullseye
        mirrors ceph_quincy_bullseye
        mirrors pmg_bullseye
        mirrors ceph_reef_bookworm
        mirrors ceph_pacific_bullseye
        mirrors debian_bookworm
        mirrors debian_bookworm_updates
        mirrors debian_bookworm_security
        mirrors debian_bullseye
        mirrors debian_bullseye_updates
        mirrors debian_bullseye_security
        mirrors pve_bookworm
        mirrors pve_bullseye
        mountpoint /media/NAS
        sync false
        verify false

Last sync :

Code: 
Stats: 12 new files (26922404b), re-used 3686 existing files (99.68% re-used)..
Skipped downloading 48 packages totalling 13982987132b

Rotating temp. snapshot in-place: "2024-11-21T05:00:50Z.tmp" -> "2024-11-21T05:00:50Z"

SUMMARY:
debian_bookworm_security: OK
debian_bookworm_updates: OK
ceph_pacific_bullseye: OK
ceph_quincy_bookworm: OK
debian_bullseye_security: OK
ceph_octopus_bullseye: OK
debian_bullseye: OK
pmg_bookworm: OK
pbs_bookworm: OK
pmg_bullseye: OK
pve_bookworm: OK
debian_bookworm: OK
ceph_quincy_bullseye: OK
pbs_bullseye: OK
ceph_reef_bookworm: OK
debian_bullseye_updates: OK
pve_bullseye: OK
Done on /var/lib/proxmox-offline-mirror/mirrors/latest/.

Thanks.
 
did you also sync your medium afterwards? I assume your PVE install is pointed at that..
 
The medium synchronize only on saturday : repository server sync from monday to friday, medium sync on saturday (we don't need more for it).

At this moment, only test machines are using local repository.
 
I am fairly certain that either your medium is outdated, or your PVE system is pointing at an old snapshot on it..
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back