[SOLVED] cephadm: No module named 'remoto' after PVE update

lesicnik1

Member
Jan 19, 2022
4
1
8
27
Hello,

I recently updated all three of our PVE hosts, that are also in a ceph cluster. During the update Ceph was updated to 16.2.9, however now I'm getting 3 warning for the ceph cluster.

Module 'cephadm' has failed dependency: loading remoto library:No module named 'remoto'

3 stray host(s) with 21 daemon(s) not managed by cephadm

21 stray daemon(s) not managed by cephadm

I've tested with the Python shell and I can import remoto as well as execute a command on another server.
I tried running pip install --upgrade remoto, but nothing was updated. pip list reports that remoto 1.2.1 is installed.

PVE Package versions:
Code:
proxmox-ve: 7.2-1 (running kernel: 5.15.64-1-pve)
pve-manager: 7.2-11 (running version: 7.2-11/b76d3178)
pve-kernel-5.15: 7.2-13
pve-kernel-helper: 7.2-13
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.15.30-2-pve: 5.15.30-3
ceph: 16.2.9-pve1
ceph-fuse: 16.2.9-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-3
libpve-guest-common-perl: 4.1-4
libpve-http-server-perl: 4.1-4
libpve-storage-perl: 7.2-10
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
openvswitch-switch: 2.15.0+ds1-2+deb11u1
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-3
pve-docs: 7.2-2
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-6
pve-firmware: 3.5-6
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.0.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1

Full output of pip list:

Code:
Package          Version
---------------- --------------
bcrypt           3.1.7
beautifulsoup4   4.9.3
ceph             1.0.0
ceph-volume      1.0.0
cephfs           2.0.0
certifi          2020.6.20
chardet          4.0.0
CherryPy         8.9.1
crit             0.0.1
cryptography     3.3.2
execnet          1.9.0
httplib2         0.18.1
idna             2.10
Jinja2           2.11.3
jsonpatch        1.25
jsonpointer      2.0
logutils         0.3.3
Mako             1.1.3
MarkupSafe       1.1.1
Paste            3.5.0
PasteDeploy      2.1.1
pecan            1.3.3
pip              20.3.4
prettytable      0.7.2
protobuf         3.12.4
pycurl           7.43.0.6
PyJWT            1.7.1
pyOpenSSL        20.0.1
PySimpleSOAP     1.16.2
python-apt       2.2.1
python-dateutil  2.8.1
python-debian    0.1.39
python-debianbts 3.1.0
PyYAML           5.3.1
rados            2.0.0
rbd              2.0.0
remoto           1.2.1
reportbug        7.10.3+deb11u1
repoze.lru       0.7
requests         2.25.1
rgw              2.0.0
Routes           2.5.1
setuptools       52.0.0
simplegeneric    0.8.1
six              1.16.0
soupsieve        2.2.1
Tempita          0.5.2
urllib3          1.26.5
waitress         1.4.4
WebOb            1.8.6
WebTest          2.0.35
Werkzeug         1.0.1
wheel            0.34.2
 
AFAIK Proxmox does not use cephadm to deploy its Ceph cluster.

Have you tried to disable this MGR module?
Looking a bit, seems like cephadm is a management layer, which I guess PVE does itself.

Would disabling cephadm be OK with regards to the cluster? I wouldn't like our environment to go down :)
 
As mentioned, cephadm is the current deployment and management provided by Ceph itself. It is not compatible if you deploy Ceph via Proxmox VE.

I assume that somehow the ceph-mgr-cephadm package was installed.
You can check it by running apt search ceph-mgr-cephadm and see if you have a [installed] at the end of the line.
 
  • Like
Reactions: lesicnik1
As mentioned, cephadm is the current deployment and management provided by Ceph itself. It is not compatible if you deploy Ceph via Proxmox VE.

I assume that somehow the ceph-mgr-cephadm package was installed.
You can check it by running apt search ceph-mgr-cephadm and see if you have a [installed] at the end of the line.
ceph-mgr-cephadm is indeed installed. Strangely enough, not installed on my other 3 server cluster (they are more or less identical in setup)

Would uninstalling it cause any issues?
 
Shouldn't be an issue. If you get those errors from cephadm, I assume that the active MON is currently the node with the package installed. If you run ceph mgr module ls and search the output, you should see cephadm. If you stop that MON and wait until one of the other nodes, without the cephadm package, become active, the same list of modules should not have cephadm listed.
 
Last edited:
  • Like
Reactions: lesicnik1
Shouldn't be an issue. If you get those errors from cephadm, I assume that the active MON is currently the node with the package installed. If you run ceph mgr module ls and search the output, you should see cephadm. If you stop that MON and wait until one of the other nodes, without the cephadm package, become active, the same list of modules should not have cephadm listed.
Thanks for your assistance, I've disabled the cephadm mgr manager module and everything seems to still be ok.
 
  • Like
Reactions: gurubert
In my case, I operate ceph manual cluster without proxmox. I met above issue when i convert manual cluster to cephadm.

Bash:
cephadm --image ceph-mon@${ceph-mon-node-hostname}

Bash:
cephadm --image ceph-mgr@${ceph-mgr-node-hostname}

After I execute above two commands, "ceph mgr module enable cephadm" command run well.

[My Ceph Environment]
OS: Debian 12
Ceph: v16.2
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!