Ceph 19.2 Squid Stable Release and Ceph 17.2 Quincy soon to be EOL

t.lamprecht

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
6,476
3,604
303
South Tyrol/Italy
shop.proxmox.com
Hi Community!

The recently released Ceph 19.2 Squid is now available on all Proxmox Ceph repositories (test, no-subscription and enterprise) to install or upgrade.

Upgrades from Reef to Squid:
You can find the upgrade how-to here: https://pve.proxmox.com/wiki/Ceph_Reef_to_Squid

New Installation of Reef:
Use the updated ceph installation wizard available with the recently released pve-manager version 8.2.8 (available on the pvetest repository at time of writing).

Current State:
We ran and tested the release since a few weeks internally and found no major issue.
Ceph 18.2 Reef will stay supported until mid-2025 for the time being.

Road to Enterprise Stability:
Our further plan is to lift the preview state and provided Squid as fully supported Ceph release once we got even more test time and feedback from QA, and naturally we would be happy to hear about the observations from our great community! Once we deem the Ceph 19.2 Squid release, and it's integration into Proxmox VE fully production ready, we'll also populate the Ceph Squid enterprise repository.
As of the Proxmox VE 8.3 release Ceph 19.2 Squid is considered stable and supported.

Reminder: Old Ceph 17.2 Quincy Going EOL Soon:
Please also remember that Ceph 17.2 Pacific is going to be end of life (EOL) soon and received no update since a while already. So you should upgrade any existing Ceph Quincy setups to Ceph Reef, or soon also Ceph Squid, rather sooner than later. See the upgrade how-to: https://pve.proxmox.com/wiki/Ceph_Quincy_to_Reef.

We welcome your feedback!

edit: the title and text have been slightly adapted to clarify that Ceph 19.2 Squid is considered stable nowadays (since Proxmox VE 8.3)
 
Last edited:
  • Like
Reactions: Tomas Waldow, wbedard, Falk R. and 8 others
Nemesiz said:
Squid have problems with orchestrator and dashboard functionality. As one created issue https://tracker.ceph.com/issues/68657
In my test lab I had the same thing. I`m just saying for just in case.
Click to expand...
For completeness' sake: Proxmox VE's Ceph management stack does not depend on any of those two, but some might use them nonetheless, so thanks for posting that hint as heads up for those users.
 
Just to be sure... the pure upgrade would not enable any "Crimson" feature for now? It's just the latest version and for anything else manual configuration will be necessary?
 
itNGO said:
Just to be sure... the pure upgrade would not enable any "Crimson" feature for now? It's just the latest version and for anything else manual configuration will be necessary?
Click to expand...
No, Crimosn is a technology preview in the Ceph 19.2 Squid release and is currently not enabled on build in our packaging.
We will take a closer look at how to expose it as opt-in co-installable with ceph-osd in the future, at the moment we're more interested in ensuring Ceph 19.2 Squid gets as stable as is.
 
  • Like
Reactions: itNGO
I hope to build a separate version of Crimosn to test its performance.
 
Last edited:
We updated one of our clusters already, no issues so far (no downtime, everything correct).
 
  • Like
Reactions: t.lamprecht
This is an up-to-date (no subscription) PVE installation with PBS alongside, but without a Ceph full-installation; only the default ootb parts:

Bash: 
Listing... Done
ceph-common/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
ceph-fuse/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
libcephfs2/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
librados2/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
libradosstriper1/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
librbd1/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
librgw2/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-ceph-argparse/stable 19.2.0-pve1 all [upgradable from: 18.2.4-pve3]
python3-ceph-common/stable 19.2.0-pve1 all [upgradable from: 18.2.4-pve3]
python3-cephfs/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-rados/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-rbd/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-rgw/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
Bash: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  autoconf automake autotools-dev cpp cpp-12 gcc gcc-12 libabsl20220623 libaom3 libasan8 libatomic1 libavif15 libc-dev-bin libc-devtools libc6-dev libcc1-0 libcrypt-dev libdav1d6 libde265-0 libdeflate0 libgav1-1 libgcc-12-dev libgd3
  libgomp1 libheif1 libisl23 libitm1 libjbig0 liblerc4 liblsan0 libltdl-dev liblua5.3-dev libmpc3 libmpfr6 libncurses-dev libnsl-dev libpkgconf3 libquadmath0 librav1e0 libreadline-dev libsvtav1enc1 libtiff6 libtirpc-dev libtool
  libtool-bin libtsan2 libubsan1 libwebp7 libx265-199 libxpm4 libyuv0 linux-libc-dev lua-any lua-sec lua-socket lua5.1 luarocks m4 manpages-dev pkg-config pkgconf pkgconf-bin rpcsvc-proto unzip zip
The following packages will be upgraded:
  ceph-common ceph-fuse libcephfs2 librados2 libradosstriper1 librbd1 librgw2 python3-ceph-argparse python3-ceph-common python3-cephfs python3-rados python3-rbd python3-rgw
13 upgraded, 65 newly installed, 0 to remove and 0 not upgraded.
Need to get 106 MB of archives.
After this operation, 254 MB of additional disk space will be used.

Is it expected, that there will get 65 new packages get pulled in?
 
Hi,
Neobin said:
This is an up-to-date (no subscription) PVE installation with PBS alongside, but without a Ceph full-installation; only the default ootb parts:

Bash: 
Listing... Done
ceph-common/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
ceph-fuse/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
libcephfs2/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
librados2/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
libradosstriper1/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
librbd1/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
librgw2/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-ceph-argparse/stable 19.2.0-pve1 all [upgradable from: 18.2.4-pve3]
python3-ceph-common/stable 19.2.0-pve1 all [upgradable from: 18.2.4-pve3]
python3-cephfs/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-rados/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-rbd/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
python3-rgw/stable 19.2.0-pve1 amd64 [upgradable from: 18.2.4-pve3]
Bash: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  autoconf automake autotools-dev cpp cpp-12 gcc gcc-12 libabsl20220623 libaom3 libasan8 libatomic1 libavif15 libc-dev-bin libc-devtools libc6-dev libcc1-0 libcrypt-dev libdav1d6 libde265-0 libdeflate0 libgav1-1 libgcc-12-dev libgd3
  libgomp1 libheif1 libisl23 libitm1 libjbig0 liblerc4 liblsan0 libltdl-dev liblua5.3-dev libmpc3 libmpfr6 libncurses-dev libnsl-dev libpkgconf3 libquadmath0 librav1e0 libreadline-dev libsvtav1enc1 libtiff6 libtirpc-dev libtool
  libtool-bin libtsan2 libubsan1 libwebp7 libx265-199 libxpm4 libyuv0 linux-libc-dev lua-any lua-sec lua-socket lua5.1 luarocks m4 manpages-dev pkg-config pkgconf pkgconf-bin rpcsvc-proto unzip zip
The following packages will be upgraded:
  ceph-common ceph-fuse libcephfs2 librados2 libradosstriper1 librbd1 librgw2 python3-ceph-argparse python3-ceph-common python3-cephfs python3-rados python3-rbd python3-rgw
13 upgraded, 65 newly installed, 0 to remove and 0 not upgraded.
Need to get 106 MB of archives.
After this operation, 254 MB of additional disk space will be used.

Is it expected, that there will get 65 new packages get pulled in?
Click to expand...
thanks for the report! What if you use --no-install-recommends?

From a quick glance, it seems like the "culprit" is librgw2 -Depends-> liblua5.3-dev -Recommends-> libtool-bin
Code: 
root@pve8a1 ~ # aptitude why autoconf
i   ceph-common   Depends    python3-rgw (= 19.2.0-pve1)
i A python3-rgw   Depends    librgw2 (>= 19.2.0-pve1)  
i A librgw2       Depends    liblua5.3-dev              
i A liblua5.3-dev Recommends libtool-bin                
i A libtool-bin   Depends    libtool (>= 2.4.7-7~deb12u1)
i A libtool       Recommends libltdl-dev                
i A libltdl-dev   Depends    automake-1.16              
i A automake      Provides   automake-1.16              
i A automake      Depends    autoconf
 
  • Like
Reactions: Neobin
This one failed for me ( command from: https://pve.proxmox.com/wiki/Ceph_Reef_to_Squid)

Code: 
root@training2:~# ceph telemetry on
Error EPERM: Telemetry data is licensed under the Community Data License Agreement - Sharing - Version 1.0 (https://cdla.io/sharing-1-0/).
To enable, add '--license sharing-1-0' to the 'ceph telemetry on' command.

It worked with:

Code: 
root@training2:~# ceph telemetry on --license sharing-1-0
Telemetry is on.
Some channels are disabled, please enable with:
`ceph telemetry enable channel perf`
 
I also noticed that if you do a fresh install, you cant find the repository in the web-ui.
It this supposed to be so?

Code: 
root@pve1:~# pveceph install --repository no-subscription --version
quincy  reef


1730122235469.png
1730122079466.png
 
Last edited:
jsterr said:
I also noticed that if you do a fresh install, you cant find the repository in the web-ui.
Click to expand...
From the first post:
New Installation of Reef:
Use the updated ceph installation wizard available with the recently released pve-manager version 8.2.8 (available on the pvetest repository at time of writing).
Click to expand...

Needs new pve-manager 8.2.8 which is in pvetest only for now.
 
  • Like
Reactions: jsterr
fiona said:
Hi,

thanks for the report! What if you use --no-install-recommends?

From a quick glance, it seems like the "culprit" is librgw2 -Depends-> liblua5.3-dev -Recommends-> libtool-bin
Code: 
root@pve8a1 ~ # aptitude why autoconf
i   ceph-common   Depends    python3-rgw (= 19.2.0-pve1)
i A python3-rgw   Depends    librgw2 (>= 19.2.0-pve1) 
i A librgw2       Depends    liblua5.3-dev             
i A liblua5.3-dev Recommends libtool-bin               
i A libtool-bin   Depends    libtool (>= 2.4.7-7~deb12u1)
i A libtool       Recommends libltdl-dev               
i A libltdl-dev   Depends    automake-1.16             
i A automake      Provides   automake-1.16             
i A automake      Depends    autoconf
Click to expand...

Thank you. This gets the count down quite a bit:

Bash: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  libc-dev-bin libc6-dev libcrypt-dev liblua5.3-dev libncurses-dev libnsl-dev libreadline-dev libtirpc-dev linux-libc-dev lua-any lua5.1 luarocks rpcsvc-proto unzip zip
The following packages will be upgraded:
  ceph-common ceph-fuse libcephfs2 librados2 libradosstriper1 librbd1 librgw2 python3-ceph-argparse python3-ceph-common python3-cephfs python3-rados python3-rbd python3-rgw
13 upgraded, 15 newly installed, 0 to remove and 0 not upgraded.
Need to get 52.6 MB of archives.
After this operation, 53.9 MB of additional disk space will be used.

As you stated, the dependencies from librgw2 of liblua5.3-dev and luarocks(?) seem to be it:

Bash: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
librgw2
  Depends: liblua5.3-dev [NotInstalled]
  Depends: luarocks [NotInstalled]
liblua5.3-dev
  Depends: libc-dev [NotInstalled]
  Depends: libc6-dev [NotInstalled]
  Depends: libreadline-dev [NotInstalled]
libc-dev
libc6-dev
  Depends: libc-dev-bin (= 2.36-9+deb12u8) [NotInstalled]
  Depends: libcrypt-dev [NotInstalled]
  Depends: libnsl-dev [NotInstalled]
  Depends: linux-libc-dev [NotInstalled]
  Depends: rpcsvc-proto [NotInstalled]
libc-dev-bin
libcrypt-dev
libnsl-dev
  Depends: libtirpc-dev [NotInstalled]
libtirpc-dev
linux-libc-dev
rpcsvc-proto
libreadline-dev
  Depends: install-info [NotInstalled]
  Depends: libncurses-dev [NotInstalled]
install-info
libncurses-dev
  Depends: libc-dev [NotInstalled]
  Depends: libc6-dev [NotInstalled]
luarocks
  Depends: liblua5.1-dev [NotInstalled]
  Depends: liblua5.2-dev [NotInstalled]
  Depends: liblua5.3-dev [NotInstalled]
  Depends: lua-any [NotInstalled]
  Depends: lua5.1 [NotInstalled]
  Depends: lua5.2 [NotInstalled]
  Depends: lua5.3 [NotInstalled]
  Depends: unzip [NotInstalled]
  Depends: zip [NotInstalled]
liblua5.1-dev
liblua5.2-dev
  Depends: libc-dev [NotInstalled]
  Depends: libc6-dev [NotInstalled]
  Depends: liblua5.2-0 (= 5.2.4-3) [NotInstalled]
  Depends: libreadline-dev [NotInstalled]
liblua5.2-0
lua-any
  Depends: lua [NotInstalled]
  Depends: lua5.1 [NotInstalled]
lua
lua5.1
lua5.2
lua5.3
unzip
zip
 
Neobin said:
Is it expected, that there will get 65 new packages get pulled in?
Click to expand...
Neobin said:
As you stated, the dependencies from librgw2 of liblua5.3-dev and luarocks(?) seem to be it:
Click to expand...

This should be fixed now with version 19.2.0-pve2, which we just uploaded to the test/no-subscription ceph-squid repo.
Many thanks again for bringing this to our attention, I also created a pull request upstream so that the broader Ceph ecosystem can profit from your findings.
 
  • Like
Reactions: ucholak and Neobin
t.lamprecht said:
This should be fixed now with version 19.2.0-pve2, which we just uploaded to the test/no-subscription ceph-squid repo.
Many thanks again for bringing this to our attention, I also created a pull request upstream so that the broader Ceph ecosystem can profit from your findings.
Click to expand...

Thank you. Currently I am seeing the new versions only in the test repository. Not at all that I am in a rush, but wanted to mention it, since it seems as not intended? :)
 
t.lamprecht said:
This should be fixed now with version 19.2.0-pve2, which we just uploaded to the test/no-subscription ceph-squid repo.
Many thanks again for bringing this to our attention, I also created a pull request upstream so that the broader Ceph ecosystem can profit from your findings.
Click to expand...
t.lamprecht said:
It was imported to no-subscription but not yet synced to the CDN, I triggered the sync now.
Click to expand...

Thank you very much. :)

Now it is all as I would expect it:
Bash: 
The following packages will be upgraded:
  ceph-common ceph-fuse libcephfs2 librados2 libradosstriper1 librbd1 librgw2 python3-ceph-argparse python3-ceph-common python3-cephfs python3-rados python3-rbd python3-rgw
13 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
 
  • Like
Reactions: wbedard, t.lamprecht and ucholak
Tiny oversight in the label (18 -> 19):
Bash: 
http://download.proxmox.com/debian/ceph-squid bookworm/no-subscription amd64 Packages
release o=Proxmox,a=stable,n=bookworm,l=Proxmox Ceph 18 Squid Debian Repository,c=no-subscription,b=amd64
origin download.proxmox.com
 
You must log in or register to reply here.
Top Bottom