POM: Mirror config won't download packages

P

pinknetworkguru

Member
Nov 10, 2021
4
1
8
Hello, I have a PVE host running 7.4-16. I have installed proxmox-offline-mirror version 0.5.2. I am trying to mirror the MongoDB repo with the following config:

Code: 
proxmox-offline-mirror config mirror add \
  --id mongodb-server-6 \
  --architectures amd64 \
  --architectures all \
  --repository 'deb https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse' \
  --key-path /etc/apt/trusted.gpg.d/mongodb-server-6.0.asc \
  --sync true \
  --verify true \
  --ignore-errors true \
  --weak-crypto allow-sha1=1 \
  --base-dir /data1

However, when I try to create a snapshot, it doesn't download any packages:

Code: 
root@pom:~# proxmox-offline-mirror mirror snapshot create mongodb-server-6
Fetching Release/Release.gpg files
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release.gpg'..
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Success
Parsing Release..
'Release' file has 3 referenced files..
Fetching InRelease file
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease'..
InRelease fetch failure: https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease: status code 404
Skipping multiverse/binary-s390x/Packages.gz
Skipping multiverse/binary-arm64/Packages.gz

Component 'multiverse'
    "multiverse/binary-amd64/Packages": Packages("amd64", None)
    "multiverse/binary-amd64/Packages.gz": Packages("amd64", Some(Gzip))
Component references count: 1
Component indices size: 246687
Total indices count: 1
Total indices size: 246687
Skipped 2 references


Fetching indices for component 'multiverse'
    Fetching 'multiverse/binary-amd64/Packages'..
    0 packages totalling 0
Progress: 0 new files (0b), re-used 1 existing files (100.00% re-used)..
Total deb size for component: 0
Total dsc size for component: 0
Total deb size: 0

Fetching multiverse packages..

multiverse/binary-amd64/Packages - no files, skipping.

Stats: 1 new files (3960b), re-used 1 existing files (50.00% re-used)..
Skipped downloading 0 packages totalling 0b

Rotating temp. snapshot in-place: "2023-07-20T20:27:59Z.tmp" -> "2023-07-20T20:27:59Z"

It seems to download the Packages file, but it doesn't seem to parse any actual package content. Am I doing something wrong?
 
I've been tinkering with this some more and I can't seem to figure out what I'm missing. When I download the Mongo packages file manually and look at it it has all the content I believe it should have. POM just seems unhappy with, or somehow unable to parse, the contents.
 
I tried again, removing "all" from the architectures list and just using amd64 instead. With a config like this:
Code: 
proxmox-offline-mirror config mirror add \
  --id mongodb-server-6 \
  --architectures amd64 \
  --repository 'deb https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse' \
  --key-path /etc/apt/trusted.gpg.d/mongodb-server-6.0.asc \
  --sync true \
  --verify true \
  --ignore-errors true \
  --weak-crypto allow-sha1=1 \
  --base-dir /data1

I get the following output:
Code: 
root@pom:~# proxmox-offline-mirror mirror snapshot create mongodb-server-6
Fetching Release/Release.gpg files
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release.gpg'..
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Success
Parsing Release..
'Release' file has 3 referenced files..
Fetching InRelease file
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease'..
InRelease fetch failure: https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/InRelease: status code 404
Skipping multiverse/binary-s390x/Packages.gz
Skipping multiverse/binary-arm64/Packages.gz

Component 'multiverse'
    "multiverse/binary-amd64/Packages": Packages("amd64", None)
    "multiverse/binary-amd64/Packages.gz": Packages("amd64", Some(Gzip))
Component references count: 1
Component indices size: 249205
Total indices count: 1
Total indices size: 249205
Skipped 2 references


Fetching indices for component 'multiverse'
    Fetching 'multiverse/binary-amd64/Packages'..
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/Packages.gz'..
    1 packages totalling 50734252
Progress: 1 new files (27144b), re-used 0 existing files (0.00% re-used)..
Total deb size for component: 50734252
Total dsc size for component: 0
Total deb size: 50734252

Fetching multiverse packages..

multiverse/binary-amd64/Packages - 1 total file(s)
-> GET 'https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/mongodb-database-tools_100.7.4_amd64.deb'..
    Progress: 1 new files (50734252b), re-used 0 existing files (0.00% re-used)..
    Progress: 1 new files (50734252b), re-used 0 existing files (0.00% re-used)..

Stats: 3 new files (50765356b), re-used 0 existing files (0.00% re-used)..
Skipped downloading 0 packages totalling 0b

Rotating temp. snapshot in-place: "2023-07-24T16:22:56Z.tmp" -> "2023-07-24T16:22:56Z"

So, now it downloads the first package in the package list. But, nothing beyond it. Is that because that package has a fairly lengthy description in the package list, maybe?

https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/Packages

The package listing in question:
Code: 
Package: mongodb-database-tools
Architecture: amd64
Version: 100.7.4
Priority: optional
Section: database
Maintainer: MongoDB Connectors Team <database-tools-packaging@mongodb.com>
Provides: mongodb-database-tools
Depends: libc6, libgssapi-krb5-2, libkrb5-3, libk5crypto3, libcomerr2, libkrb5support0, libkeyutils1
Conflicts: mongodb-database-tools
Breaks: mongodb-org-tools (<= 4.3.2), mongodb-org-tools-unstable (<= 4.3.2), mongodb-enterprise-tools (<= 4.3.2), mongodb-enterprise-tools-unstable (<= 4.3.2)
Replaces: mongodb-database-tools (<= 100.7.4), mongodb-org-tools (<= 4.3.2), mongodb-org-tools-unstable (<= 4.3.2), mongodb-enterprise-tools (<= 4.3.2), mongodb-enterprise-tools-unstable (<= 4.3.2)
Filename: dists/jammy/mongodb-org/6.0/multiverse/binary-amd64/mongodb-database-tools_100.7.4_amd64.deb
Size: 50734252
MD5sum: 7717b1a4338f9f6c2b8753ef985529fe
SHA1: c41555a021dfc5e01bb4bfcf511a975b44522a40
SHA256: 3d5827052a98cac72dcd540e4c94415d2f87809e0afcb064910eab6c334b2aaa
SHA512: 8ff46f3e91a14168a0f3f91e42a66fda685ec1aca134098dd4897596999df27e3c66b2d86d43e4ebb0f94212ded821d97cbab2f6487d42f26f12004fc6b1769d
Homepage: http://www.mongodb.com
Description: mongodb-database-tools package provides tools for working with the MongoDB server:
 *bsondump - display BSON files in a human-readable format
 *mongoimport - Convert data from JSON, TSV or CSV and insert them into a collection
 *mongoexport - Write an existing collection to CSV or JSON format
 *mongodump/mongorestore - Dump MongoDB backups to disk in .BSON format,
    or restore them to a live database
 *mongostat - Monitor live MongoDB servers, replica sets, or sharded clusters
 *mongofiles - Read, write, delete, or update files in GridFS
    (see: http://docs.mongodb.org/manual/core/gridfs/)
 *mongotop - Monitor read/write activity on a mongo server
 
Hi

I encountered similar issue with TimescaleDB repo. I am using also proxmox-offline-mirror version 0.5.2

Mirror setup in /etc/proxmox-offline-mirror.cfg:
Code: 
mirror: timescaledb-bullseye
    architectures amd64
    architectures all
    base-dir /repo
    ignore-errors false
    key-path /etc/apt/trusted.gpg.d/timescaledb.gpg
    repository deb https://packagecloud.io/timescale/timescaledb/debian bullseye main
    sync false
    verify true
    weak-crypto allow-sha1=1

Trying to create the snapshot doesn't download any actual packages:
Code: 
Fetching Release/Release.gpg files
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/Release.gpg'..
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
Success
Parsing Release..
'Release' file has 70 referenced files..
Fetching InRelease file
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/InRelease'..
Verifying 'InRelease' signature using provided repository key..
27169 bytes verified
Success
Parsing InRelease..
'InRelease' file has 70 referenced files..
Skipping Contents-arm64.gz
Skipping Contents-mipsel.bz2
Skipping main/binary-m32/Packages
Skipping main/binary-ppc64/Packages.gz
Skipping Contents-ia64.bz2
Skipping main/binary-armel/Packages.gz
Skipping main/binary-mipsel/Packages.gz
Skipping Contents-sparc.bz2
Skipping main/binary-arm/Packages.bz2
Skipping main/binary-armhf/Packages.bz2
Skipping Contents-ppc64.gz
Skipping Contents-s390.gz
Skipping main/binary-musl-linux-amd64/Packages.gz
Skipping Contents-s390x.gz
Skipping main/binary-i386/Packages.gz
Skipping Contents-m32.gz
Skipping Contents-mips.bz2
Skipping Contents-avr32.gz
Skipping Contents-alpha.gz
Skipping Contents-mips.gz
Skipping Contents-s390.bz2
Skipping Contents-arm.gz
Skipping Contents-ia64.gz
Skipping main/binary-ppc64el/Packages.bz2
Skipping Contents-mips64el.bz2
Skipping Contents-amd64.gz
Skipping Contents-musl-linux-amd64.gz
Skipping Contents-m68k.bz2
Skipping main/binary-s390/Packages
Skipping Contents-i386.gz
Skipping main/source/Sources.bz2
Skipping Contents-sh.gz
Skipping Contents-m32.bz2
Skipping main/binary-sh/Packages.gz
Skipping Contents-m68k.gz
Skipping Contents-s390x.bz2
Skipping main/binary-avr32/Packages.bz2
Skipping main/binary-m68k/Packages.bz2
Skipping main/binary-hppa/Packages.bz2
Skipping Contents-armel.gz
Skipping Contents-mipsel.gz
Skipping main/binary-powerpc/Packages
Skipping Contents-armhf.bz2
Skipping Contents-powerpc.bz2
Skipping main/binary-s390x/Packages.gz
Skipping Contents-sparc.gz
Skipping main/binary-alpha/Packages
Skipping Contents-armel.bz2
Skipping Contents-hppa.bz2
Skipping Contents-arm.bz2
Skipping Contents-powerpc.gz
Skipping Contents-armhf.gz
Skipping main/binary-ia64/Packages.bz2
Skipping Contents-ppc64el.bz2
Skipping Contents-amd64.bz2
Skipping Contents-arm64.bz2
Skipping Contents-mips64el.gz
Skipping Contents-i386.bz2
Skipping main/binary-mips/Packages
Skipping Contents-ppc64.bz2
Skipping Contents-avr32.bz2
Skipping main/binary-arm64/Packages
Skipping Contents-sh.bz2
Skipping Contents-hppa.gz
Skipping main/binary-mips64el/Packages.bz2
Skipping Contents-ppc64el.gz
Skipping Contents-alpha.bz2
Skipping Contents-musl-linux-amd64.bz2
Skipping main/binary-sparc/Packages.bz2

Component 'main'
    "main/binary-amd64/Packages": Packages("amd64", None)
    "main/binary-amd64/Packages.bz2": Packages("amd64", Some(Bzip2))
    "main/binary-amd64/Packages.gz": Packages("amd64", Some(Gzip))
Component references count: 1
Component indices size: 904734
Total indices count: 1
Total indices size: 904734
Skipped 69 references


Fetching indices for component 'main'
    Fetching 'main/binary-amd64/Packages'..
-> GET 'https://packagecloud.io/timescale/timescaledb/debian/dists/bullseye/main/binary-amd64/by-hash/SHA256/12c3c5f76932733f67869898315d4cf46e2635dd8f422be3c7e6a13e31774236'..
    0 packages totalling 0
Progress: 1 new files (132916b), re-used 1 existing files (50.00% re-used)..
Total deb size for component: 0
Total dsc size for component: 0
Total deb size: 0

Fetching main packages..

main/binary-amd64/Packages - no files, skipping.

Stats: 3 new files (188971b), re-used 1 existing files (25.00% re-used)..
Skipped downloading 0 packages totalling 0b

Rotating temp. snapshot in-place: "2024-01-04T16:23:07Z.tmp" -> "2024-01-04T16:23:07Z"

Did somebody find any resolution?

Thanks
 
You must log in or register to reply here.
Top Bottom
Back