Proxmox Offline Mirror - Debian Repository needed?

[roggeb]

Hi,

I set um the Proxmox Offline Mirror successfully, and connected my nodes with it.

My question is fairly simple: To keep my nodes updated (and most importantly all security holes patched), do I need a snapshot of the Debian repository, too?

Right now I only snapshotted (meaning syncd and downloaded) the PVE and Ceph repositories, thinking that would be enough for the future. But now I am in doubt. I guess the newest kernel version would only be in the Debian repository? (Excuse my naiveté, I have been a purely Windows admin and user before.)
I am asking, because the Debian repo is fairly big and I would prefer to skip that.

Here are the options the PMO gives me for selecting repositories, when doing the guided setup:

Select distro to mirror
   0) Proxmox VE
   1) Proxmox Backup Server
   2) Proxmox Mail Gateway
   3) Proxmox Ceph
   4) Debian

Also: why can't I put the PVE, Ceph and Debian repo into one snapshot? It says my input "034" is not a valid choice. Whereas "03" is.

Thank you in advance.

 

[fabian]

you also need the matching Debian repository, just like a regular PVE system needs to have that configured. if you use the guided mode to setup a PVE mirror, it will ask you whether you want to mirror the corresponding Debian repositories as well. you can filter (by section or package name) to reduce the scope of the mirroring if space usage is a concern - but note that Debian stable gets a lot less updates than PVE itself, so once you have a mirror, it shouldn't be too expensive to keep it up to date.

Also: why can't I put the PVE, Ceph and Debian repo into one snapshot? It says my input "034" is not a valid choice. Whereas "03" is.

well, "03" is "3" - this is a single choice prompt each repository is snapshotted on its on anyway, we cannot force the other repositories to freeze while POM is busy mirroring the first one after all.. I guess we could add an option to mirror all Proxmox products + Debian.

 

[roggeb]

Thank you for your very fast response, fabian.

Cool, then I will snapshot the Debian repo as well. But could it be possible that it is way more than 5 TB as I see here: https://www.debian.org/mirror/size ? Hard sell to the storage team.
The PMO manual suggests to skip the "linux image", "games" and "debug" packages. Games I get, but is the "linux image" package not essential for the kernel like it says in the PMO manual ("filter Debian linux kernel image packages") (again, Linux noob at work)? Also debug information seems important.
Can you point me in the right direction of what to skip exactly, that is irrelevant for a fully functioning PVE operation?

Edit: Concerning the numbers input in the guided installation: If I do a "apt update" and then a "apt search pve" on a node connected to the PMO I do get PVE packages. So the input of "03" must have also included PVE (0).
Just as an example:

pve-firmware/now 3.7-1 all [installed,local]
  Binary firmware code for the pve-kernel

 

[fabian]

a full snapshot of a single Debian release (like "Bookworm") is something like 100-150G, depending on your filter settings. note that you only need architecture all and amd64 packages, as opposed to Debian itself, which supports a much wider range of architectures (and therefore also needs a lot more space a full mirror of Debian will also contain things like installer files that are not mirrored by POM).

"linux-image" will filter out the Debian kernel image packages, since we ship our own.
"debug" is needed for debugging, obviously. if you don't expect to to do that, you can skip them (they can get quite big!)
pve-firmware (and some other packages) contains "pve" in the package name, but are actually relevant for all our products. this has historic reasons, and where easily doable, we try to phase those out to a more generic "promox-" prefix.

 

[roggeb]

Great. Thanks for shining a light on this topic.

 

[DynFi User]

By any chances do you have an optimized version of the "--skip-sections" available…
I could really use that since the repo is filling up my laptop which has limited ressources…

Thx !

 

[fabian]

what do you mean with "optimized"? you can already chose what you want to mirror, nobody but you will be able to tell what you need

 

[DynFi User]

Well, among the 90G of Debian stuff, I thought that maybe you had a more optimized tuning of the "--skip-sections"
This is mostly what I am looking for.

This is what's provided in your example, but we still have 90G with a lot of Debian unused packages.

--skip-sections 'games' \
--skip-sections 'debug' \
--skip-packages 'linux-image-*' \

 

[fabian]

you need to decide what you don't need - since you can in theory install almost any Debian package on a PVE/PBS system..

there's different solutions if all you want to do is cache actually used package files for local access (apt-cacher-ng is the most popular one).

 

[DynFi User]

Well we want the bare minimum to have a properly functioning PVE.
Nothing more.

 

[fabian]

there is no such static list of filters - PVE is not static, new dependencies might be required in a future upgrade.. that's why we have a "sane" set of default exclusions of things we know we never need (well, debug can be needed for debugging, but not for regular operations), knowing that this still means mirroring packages that most PVE systems will never have installed..

 

[DynFi User]

Yes, that's what I thought.

At some point might be interesting to offer a mirror with only sanitized Debian packages.
For PoM customers, It will make sense and greatly speed-up the install process (copying 80.000+ packages takes a while).

Thx anyway for answering.