POM: Mirror config won't download packages

Nov 10, 2021
4
1
8
Hello, I have a PVE host running 7.4-16. I have installed proxmox-offline-mirror version 0.5.2. I am trying to mirror the MongoDB repo with the following config:

Code:
proxmox-offline-mirror config mirror add \
  --id mongodb-server-6 \
  --architectures amd64 \
  --architectures all \
  --repository 'deb https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse' \
  --key-path /etc/apt/trusted.gpg.d/mongodb-server-6.0.asc \
  --sync true \
  --verify true \
  --ignore-errors true \
  --weak-crypto allow-sha1=1 \
  --base-dir /data1

However, when I try to create a snapshot, it doesn't download any packages:

Code:
root@pom:~# proxmox-offline-mirror mirror snapshot create mongodb-server-6
Fetching Release/Release.gpg files
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release.gpg'..
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Success
Parsing Release..
'Release' file has 3 referenced files..
Fetching InRelease file
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease'..
InRelease fetch failure: https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease: status code 404
Skipping multiverse/binary-s390x/Packages.gz
Skipping multiverse/binary-arm64/Packages.gz

Component 'multiverse'
    "multiverse/binary-amd64/Packages": Packages("amd64", None)
    "multiverse/binary-amd64/Packages.gz": Packages("amd64", Some(Gzip))
Component references count: 1
Component indices size: 246687
Total indices count: 1
Total indices size: 246687
Skipped 2 references


Fetching indices for component 'multiverse'
    Fetching 'multiverse/binary-amd64/Packages'..
    0 packages totalling 0
Progress: 0 new files (0b), re-used 1 existing files (100.00% re-used)..
Total deb size for component: 0
Total dsc size for component: 0
Total deb size: 0

Fetching multiverse packages..

multiverse/binary-amd64/Packages - no files, skipping.

Stats: 1 new files (3960b), re-used 1 existing files (50.00% re-used)..
Skipped downloading 0 packages totalling 0b

Rotating temp. snapshot in-place: "2023-07-20T20:27:59Z.tmp" -> "2023-07-20T20:27:59Z"

It seems to download the Packages file, but it doesn't seem to parse any actual package content. Am I doing something wrong?
 
I've been tinkering with this some more and I can't seem to figure out what I'm missing. When I download the Mongo packages file manually and look at it it has all the content I believe it should have. POM just seems unhappy with, or somehow unable to parse, the contents.
 
I tried again, removing "all" from the architectures list and just using amd64 instead. With a config like this:
Code:
proxmox-offline-mirror config mirror add \
  --id mongodb-server-6 \
  --architectures amd64 \
  --repository 'deb https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse' \
  --key-path /etc/apt/trusted.gpg.d/mongodb-server-6.0.asc \
  --sync true \
  --verify true \
  --ignore-errors true \
  --weak-crypto allow-sha1=1 \
  --base-dir /data1

I get the following output:
Code:
root@pom:~# proxmox-offline-mirror mirror snapshot create mongodb-server-6
Fetching Release/Release.gpg files
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release.gpg'..
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Success
Parsing Release..
'Release' file has 3 referenced files..
Fetching InRelease file
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease'..
InRelease fetch failure: https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease: status code 404
Skipping multiverse/binary-s390x/Packages.gz
Skipping multiverse/binary-arm64/Packages.gz

Component 'multiverse'
    "multiverse/binary-amd64/Packages": Packages("amd64", None)
    "multiverse/binary-amd64/Packages.gz": Packages("amd64", Some(Gzip))
Component references count: 1
Component indices size: 249205
Total indices count: 1
Total indices size: 249205
Skipped 2 references


Fetching indices for component 'multiverse'
    Fetching 'multiverse/binary-amd64/Packages'..
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/Packages.gz'..
    1 packages totalling 50734252
Progress: 1 new files (27144b), re-used 0 existing files (0.00% re-used)..
Total deb size for component: 50734252
Total dsc size for component: 0
Total deb size: 50734252

Fetching multiverse packages..

multiverse/binary-amd64/Packages - 1 total file(s)
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/mongodb-database-tools_100.7.4_amd64.deb'..
    Progress: 1 new files (50734252b), re-used 0 existing files (0.00% re-used)..
    Progress: 1 new files (50734252b), re-used 0 existing files (0.00% re-used)..

Stats: 3 new files (50765356b), re-used 0 existing files (0.00% re-used)..
Skipped downloading 0 packages totalling 0b

Rotating temp. snapshot in-place: "2023-07-24T16:22:56Z.tmp" -> "2023-07-24T16:22:56Z"

So, now it downloads the first package in the package list. But, nothing beyond it. Is that because that package has a fairly lengthy description in the package list, maybe?

https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/Packages

The package listing in question:
Code:
Package: mongodb-database-tools
Architecture: amd64
Version: 100.7.4
Priority: optional
Section: database
Maintainer: MongoDB Connectors Team <database-tools-packaging@mongodb.com>
Provides: mongodb-database-tools
Depends: libc6, libgssapi-krb5-2, libkrb5-3, libk5crypto3, libcomerr2, libkrb5support0, libkeyutils1
Conflicts: mongodb-database-tools
Breaks: mongodb-org-tools (<= 4.3.2), mongodb-org-tools-unstable (<= 4.3.2), mongodb-enterprise-tools (<= 4.3.2), mongodb-enterprise-tools-unstable (<= 4.3.2)
Replaces: mongodb-database-tools (<= 100.7.4), mongodb-org-tools (<= 4.3.2), mongodb-org-tools-unstable (<= 4.3.2), mongodb-enterprise-tools (<= 4.3.2), mongodb-enterprise-tools-unstable (<= 4.3.2)
Filename: dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/mongodb-database-tools_100.7.4_amd64.deb
Size: 50734252
MD5sum: 7717b1a4338f9f6c2b8753ef985529fe
SHA1: c41555a021dfc5e01bb4bfcf511a975b44522a40
SHA256: 3d5827052a98cac72dcd540e4c94415d2f87809e0afcb064910eab6c334b2aaa
SHA512: 8ff46f3e91a14168a0f3f91e42a66fda685ec1aca134098dd4897596999df27e3c66b2d86d43e4ebb0f94212ded821d97cbab2f6487d42f26f12004fc6b1769d
Homepage: http://www.mongodb.com
Description: mongodb-database-tools package provides tools for working with the MongoDB server:
 *bsondump - display BSON files in a human-readable format
 *mongoimport - Convert data from JSON, TSV or CSV and insert them into a collection
 *mongoexport - Write an existing collection to CSV or JSON format
 *mongodump/mongorestore - Dump MongoDB backups to disk in .BSON format,
    or restore them to a live database
 *mongostat - Monitor live MongoDB servers, replica sets, or sharded clusters
 *mongofiles - Read, write, delete, or update files in GridFS
    (see: http://docs.mongodb.org/manual/core/gridfs/)
 *mongotop - Monitor read/write activity on a mongo server
 
Hi

I encountered similar issue with TimescaleDB repo. I am using also proxmox-offline-mirror version 0.5.2

Mirror setup in /etc/proxmox-offline-mirror.cfg:
Code:
mirror: timescaledb-bullseye
    architectures amd64
    architectures all
    base-dir /repo
    ignore-errors false
    key-path /etc/apt/trusted.gpg.d/timescaledb.gpg
    repository deb https://packagecloud.io/timescale/timescaledb/debian bullseye main
    sync false
    verify true
    weak-crypto allow-sha1=1

Trying to create the snapshot doesn't download any actual packages:
Code:
Fetching Release/Release.gpg files
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/Release.gpg'..
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Success
Parsing Release..
'Release' file has 70 referenced files..
Fetching InRelease file
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/InRelease'..
Verifying 'InRelease' signature using provided repository key..
27169 bytes verified
Success
Parsing InRelease..
'InRelease' file has 70 referenced files..
Skipping Contents-arm64.gz
Skipping Contents-mipsel.bz2
Skipping main/binary-m32/Packages
Skipping main/binary-ppc64/Packages.gz
Skipping Contents-ia64.bz2
Skipping main/binary-armel/Packages.gz
Skipping main/binary-mipsel/Packages.gz
Skipping Contents-sparc.bz2
Skipping main/binary-arm/Packages.bz2
Skipping main/binary-armhf/Packages.bz2
Skipping Contents-ppc64.gz
Skipping Contents-s390.gz
Skipping main/binary-musl-linux-amd64/Packages.gz
Skipping Contents-s390x.gz
Skipping main/binary-i386/Packages.gz
Skipping Contents-m32.gz
Skipping Contents-mips.bz2
Skipping Contents-avr32.gz
Skipping Contents-alpha.gz
Skipping Contents-mips.gz
Skipping Contents-s390.bz2
Skipping Contents-arm.gz
Skipping Contents-ia64.gz
Skipping main/binary-ppc64el/Packages.bz2
Skipping Contents-mips64el.bz2
Skipping Contents-amd64.gz
Skipping Contents-musl-linux-amd64.gz
Skipping Contents-m68k.bz2
Skipping main/binary-s390/Packages
Skipping Contents-i386.gz
Skipping main/source/Sources.bz2
Skipping Contents-sh.gz
Skipping Contents-m32.bz2
Skipping main/binary-sh/Packages.gz
Skipping Contents-m68k.gz
Skipping Contents-s390x.bz2
Skipping main/binary-avr32/Packages.bz2
Skipping main/binary-m68k/Packages.bz2
Skipping main/binary-hppa/Packages.bz2
Skipping Contents-armel.gz
Skipping Contents-mipsel.gz
Skipping main/binary-powerpc/Packages
Skipping Contents-armhf.bz2
Skipping Contents-powerpc.bz2
Skipping main/binary-s390x/Packages.gz
Skipping Contents-sparc.gz
Skipping main/binary-alpha/Packages
Skipping Contents-armel.bz2
Skipping Contents-hppa.bz2
Skipping Contents-arm.bz2
Skipping Contents-powerpc.gz
Skipping Contents-armhf.gz
Skipping main/binary-ia64/Packages.bz2
Skipping Contents-ppc64el.bz2
Skipping Contents-amd64.bz2
Skipping Contents-arm64.bz2
Skipping Contents-mips64el.gz
Skipping Contents-i386.bz2
Skipping main/binary-mips/Packages
Skipping Contents-ppc64.bz2
Skipping Contents-avr32.bz2
Skipping main/binary-arm64/Packages
Skipping Contents-sh.bz2
Skipping Contents-hppa.gz
Skipping main/binary-mips64el/Packages.bz2
Skipping Contents-ppc64el.gz
Skipping Contents-alpha.bz2
Skipping Contents-musl-linux-amd64.bz2
Skipping main/binary-sparc/Packages.bz2

Component 'main'
    "main/binary-amd64/Packages": Packages("amd64", None)
    "main/binary-amd64/Packages.bz2": Packages("amd64", Some(Bzip2))
    "main/binary-amd64/Packages.gz": Packages("amd64", Some(Gzip))
Component references count: 1
Component indices size: 904734
Total indices count: 1
Total indices size: 904734
Skipped 69 references


Fetching indices for component 'main'
    Fetching 'main/binary-amd64/Packages'..
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/main/binary-amd64/by-hash/SHA256/12c3c5f76932733f67869898315d4cf46e2635dd8f422be3c7e6a13e31774236'..
    0 packages totalling 0
Progress: 1 new files (132916b), re-used 1 existing files (50.00% re-used)..
Total deb size for component: 0
Total dsc size for component: 0
Total deb size: 0

Fetching main packages..

main/binary-amd64/Packages - no files, skipping.

Stats: 3 new files (188971b), re-used 1 existing files (25.00% re-used)..
Skipped downloading 0 packages totalling 0b

Rotating temp. snapshot in-place: "2024-01-04T16:23:07Z.tmp" -> "2024-01-04T16:23:07Z"

Did somebody find any resolution?

Thanks
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!