CEPH installation is failing due to a version mismatch

[Kuldeep.Roy]

I am facing an issue while installing Ceph on our 3-node Proxmox VE 9.0.11 (Trixie) cluster using the Enterprise Ceph-Quincy repository.

During installation, apt reports dependency conflicts due to mismatched package builds:

• ceph and ceph-base are at 19.2.3-pve1
• ceph-common, librados2, librbd1, and Python modules are at 19.2.3-pve2

ii ceph-common 19.2.3-pve2 amd64 common utilities to mount and interact with a ceph storage cluster
ii ceph-fuse 19.2.3-pve2 amd64 FUSE-based client for the Ceph distributed file system
ii libcephfs2 19.2.3-pve2 amd64 Ceph distributed file system client library
ii libpve-access-control 9.0.3 all Proxmox VE access control library
ii libpve-apiclient-perl 3.4.0 all Proxmox VE API client library
ii libpve-cluster-api-perl 9.0.6 all Proxmox Virtual Environment cluster Perl API modules.
ii libpve-cluster-perl 9.0.6 all Proxmox Virtual Environment cluster Perl modules.
ii libpve-common-perl 9.0.11 all Proxmox VE base library
ii libpve-guest-common-perl 6.0.2 all Proxmox VE common guest-related modules
ii libpve-http-server-perl 6.0.4 all Proxmox Asynchrounous HTTP Server Implementation
ii libpve-network-api-perl 1.1.8 all API endpoints for Proxmox VE's SDN stack
ii libpve-network-perl 1.1.8 all Proxmox VE's SDN (Software Defined Network) stack
ii libpve-notify-perl 9.0.6 all Notify helper module.
ii libpve-rs-perl 0.10.10 amd64 PVE parts which have been ported to Rust - Rust source code
ii libpve-storage-perl 9.0.13 all Proxmox VE storage management library
ii librados2 19.2.3-pve2 amd64 RADOS distributed object store client library
ii librados2-perl 1.5.0 amd64 Perl bindings for librados
ii libradosstriper1 19.2.3-pve2 amd64 RADOS striping interface
ii librbd1 19.2.3-pve2 amd64 RADOS block device client library
ii proxmox-kernel-6.14.11-4-pve-signed 6.14.11-4 amd64 Proxmox Kernel Image (signed)
ii proxmox-kernel-6.14.8-2-pve-signed 6.14.8-2 amd64 Proxmox Kernel Image (signed)
ii proxmox-ve 9.0.0 all Proxmox Virtual Environment
ii pve-cluster 9.0.6 amd64 "pmxcfs" distributed cluster filesystem for Proxmox Virtual Environment.
ii pve-container 6.0.13 all Proxmox VE Container management tool
ii pve-docs 9.0.8 all Proxmox VE Documentation
ii pve-edk2-firmware 4.2025.02-4 all edk2 based UEFI firmware modules for virtual machines
ii pve-edk2-firmware-legacy 4.2025.02-4 all edk2 based legacy 2MB UEFI firmware modules for virtual machines
ii pve-edk2-firmware-ovmf 4.2025.02-4 all edk2 based UEFI firmware modules for virtual machines
ii pve-esxi-import-tools 1.0.1 amd64 Tools to allow importing VMs from ESXi hosts
ii pve-firewall 6.0.3 amd64 Proxmox VE Firewall
ii pve-firmware 3.17-2 all Binary firmware code for the pve-kernel
ii pve-ha-manager 5.0.5 amd64 Proxmox VE HA Manager
ii pve-i18n 3.6.1 all Internationalization support for Proxmox VE
ii pve-lxc-syscalld 2.0.2 amd64 PVE LXC syscall daemon
ii pve-manager 9.0.11 all Proxmox Virtual Environment Management Tools
ii pve-nvidia-vgpu-helper 0.2.0 all Proxmox Nvidia vGPU helper script and systemd service
ii pve-qemu-kvm 10.0.2-4 amd64 Full virtualization on x86 hardware
ii pve-xtermjs 5.5.0-2 all HTML/TypeScript based fully-featured terminal for Proxmox projects
ii pve-yew-mobile-gui 0.6.2 amd64 Mobile Web UI to for the Proxmox Virtual Environment
ii pve-yew-mobile-i18n 3.6.1 all Internationalization support for Proxmox Virtual Environment (yew PWA)
ii python3-ceph-argparse 19.2.3-pve2 all Python 3 utility libraries for Ceph CLI
ii python3-ceph-common 19.2.3-pve2 all Python 3 utility libraries for Ceph
ii python3-cephfs 19.2.3-pve2 amd64 Python 3 libraries for the Ceph libcephfs library
ii python3-rados 19.2.3-pve2 amd64 Python 3 libraries for the Ceph librados library
ii python3-rbd 19.2.3-pve2 amd64 Python 3 libraries for the Ceph librbd library



This results in the following error:

update available package list
start installation
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ceph-common is already the newest version (19.2.3-pve2).
ceph-fuse is already the newest version (19.2.3-pve2).
gdisk is already the newest version (1.0.10-2).
Solving dependencies... Error!
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 ceph-base : Depends: ceph-common (= 19.2.3-pve1) but 19.2.3-pve2 is to be installed
 ceph-mgr : Depends: libsqlite3-mod-ceph (= 19.2.3-pve1) but it is not going to be installed
            Depends: librados2 (= 19.2.3-pve1) but 19.2.3-pve2 is to be installed
E: Unable to correct problems, you have held broken packages.
E: The following information from --solver 3.0 may provide additional context:
   Unable to satisfy dependencies. Reached two conflicting decisions:
   1. ceph-common:amd64=19.2.3-pve1 is not selected for install
   2. ceph-common:amd64=19.2.3-pve1 is selected as a downgrade because:
      1. ceph-mds:amd64=19.2.3-pve1 is selected for install
      2. ceph-mds:amd64=19.2.3-pve1 Depends ceph-base (= 19.2.3-pve1)
      3. ceph-base:amd64=19.2.3-pve1 Depends ceph-common (= 19.2.3-pve1)
apt failed during ceph installation (25600)

All nodes are fully up to date (apt clean && apt update), and our /etc/apt/sources.list.d entries are correctly pointing to:


deb https://enterprise.proxmox.com/debian/pve trixie enterprise
deb https://enterprise.proxmox.com/debian/ceph-quincy trixie enterprise

Can anyone please confirm if this is a known mirror synchronization issue with the Enterprise Ceph-Quincy Trixie repository? Should I wait a few hours to sync? OR I can temporary add no subscription repo to update and remove later -- this approach seems risky to me for PROD env so I am asking for confirmation.

 

[pankaj007]

Hi Kuldeep.Roy

We faced this same issue and it was was resolved finally. Explaining the issue below and the resolution.

Issue:
This is a full Ceph dependency chain issue. What’s happening:
You’re trying to downgrade some packages (e.g., ceph-common, librados2) to 19.2.3-pve1.

Few packages in your system is still pve2 by default (19.2.3-pve2), and downgrading it is blocked because other packages depend on it.
So apt is caught in a conflict loop. To fix this, you need to:
Explicitly downgrade the full Ceph stack including the required libraries/packages.

Resolution:
For the same case you mentioned above below command worked.
Include all Python bindings and libraries that are part of 19.2.3-pve1.

apt update
apt install --allow-downgrades -y \
  ceph=19.2.3-pve1 \
  ceph-common=19.2.3-pve1 \
  ceph-base=19.2.3-pve1 \
  ceph-mgr=19.2.3-pve1 \
  ceph-mon=19.2.3-pve1 \
  ceph-osd=19.2.3-pve1 \
  ceph-fuse=19.2.3-pve1 \
  libcephfs2=19.2.3-pve1 \
  librados2=19.2.3-pve1 \
  librbd1=19.2.3-pve1 \
  libradosstriper1=19.2.3-pve1 \
  librgw2=19.2.3-pve1 \
  python3-ceph-common=19.2.3-pve1 \
  python3-cephfs=19.2.3-pve1 \
  python3-ceph-argparse=19.2.3-pve1 \
  python3-rados=19.2.3-pve1 \
  python3-rbd=19.2.3-pve1 \
  python3-rgw=19.2.3-pve1

After this, all Ceph packages and dependencies should match 19.2.3-pve1.
Verify with below commands:

dpkg -l | grep ceph
dpkg -l | grep librados
dpkg -l | grep librbd
dpkg -l | grep rgw

 

[VictorSTS]

Proxmox VE 9.0.11 (Trixie) cluster using the Enterprise Ceph-Quincy repository

There is no such thing. The only supported version on PVE 9 is Ceph Squid. If this is an updated cluster from PVE8, you should have updated repos and Ceph to Squid [1].

You package list show packages version 19.2.*, not Quincy ones (17.2.*). Maybe you have both repositories configured?
I've already installed / upgraded a few PVE9 + Ceph Squid and have had no issue with dependencies.

[1] https://pve.proxmox.com/wiki/Upgrade_from_8_to_9#Update_the_Ceph_Package_Repository