[SOLVED] Ghost monitors, managers and meta data servers

bitflow

New Member
Jan 19, 2021
10
0
1
Hi. I am running Proxmox 7.0-11 with CEPH version 16.2.6. I have some trouble with the GUI Datacenter -> Ceph -> Services.

I see a few services marked as unknown for some reason that is related to CEPH.. See screenshot below.

1633527293652.png
I've tried systemctl restart pveproxy on all nodes and clearing my browser cookies but with no result. Running for example ceph mon dump shows only three monitors which is correct so I don't get where these extra ones comes from. Also checked /var/lib/ceph/[mon|mgr|mds] in the CLI and there was only one entry per service there.

Any clues on how to resolve this?
 
what does
Code:
pvesh get /cluster/ceph/status
show ?
 
I can't send this output to a file/clipboard
you can redirect the output into a file (also please add --output-format json):

Code:
pvesh get /cluster/ceph/status --output-format json > /tmp/status.txt

its then in the file /tmp/status.txt
 
you can redirect the output into a file (also please add --output-format json):

Code:
pvesh get /cluster/ceph/status --output-format json > /tmp/status.txt

its then in the file /tmp/status.txt
Thanks. Here it is.
 

Attachments

  • status.txt
    139.8 KB · Views: 6
thx, i forgot to also ask for
Code:
pvesh get /cluster/ceph/metadata --output-format json > /tmp/metadata.txt

(the gui queries both api endpoints and merges them)
 
Hi,

i had the same problem: https://forum.proxmox.com/threads/duplicated-ceph-mon-mgr-and-mds-in-ceph-overview.89374/
My fix was that only the node name should be present in /etc/hostname and not the full fqdn - maybe that helps.

Greetz
Hi.

Thank you for the suggestion!

I tried updating the hostname on the three nodes without using FQDN and then I restarted mon, mgr and mds services. Unfortunately they were still present.

I rebooted a node at the time and after that the ghost services seemed to have vanished..

Is this a bug in Proxmox or Ceph? Would be nice if we could use FQDN in `/etc/hostname`.

thx, i forgot to also ask for
Code:
pvesh get /cluster/ceph/metadata --output-format json > /tmp/metadata.txt

(the gui queries both api endpoints and merges them)
See attached. I replaced the hostnames fyi.

Looks good now.
1634038531358.png
 

Attachments

  • metadata.txt
    65.8 KB · Views: 4
Last edited:
I tried updating the hostname on the three nodes without using FQDN and then I restarted mon, mgr and mds services. Unfortunately they were still present.

I rebooted a node at the time and after that the ghost services seemed to have vanished..
did you restart/reload the browser/ui in between?
if not that could explain it, since the ui tries to keep them.. but i'll look into it if it's a bug

Is this a bug in Proxmox or Ceph? Would be nice if we could use FQDN in `/etc/hostname`.
normally you would define both:
Code:
my.own.ip.address hostname hostname.domain.tld

EDIT: in general, the nodename must resolve to a valid ip (without the fqdn)
 
Last edited:
did you restart/reload the browser/ui in between?
if not that could explain it, since the ui tries to keep them.. but i'll look into it if it's a bug


normally you would define both:
Code:
my.own.ip.address hostname hostname.domain.tld
I cleared the cache of my browser and tried to restart pveproxy between but it was after the reboot the ghost services disappeared.

Isn't that the format of the /etc/hosts? That file I haven't touch I only changed /etc/hostname for now. hostname command returns the short name and hostname --fqdn shows the full hostname as it should.
 
Isn't that the format of the /etc/hosts? That file I haven't touch I only changed /etc/hostname/ for now. hostname command returns the short name and hostname --fqdn shows the full hostname as it should.
ah yes sorry

/etc/hostname should only contain the nodename and the fqdn should be in /etc/resolv.conf

Code:
domain xyz.com
search xyz.com
nameserver some.dns.ip.address

EDIT again: for more details about how to set what, check 'man hostname'
 
Last edited:
  • Like
Reactions: bitflow
ah yes sorry

/etc/hostname should only contain the nodename and the fqdn should be in /etc/resolv.conf

Code:
domain xyz.com
search xyz.com
nameserver some.dns.ip.address

EDIT again: for more details about how to set what, check 'man hostname'
Thank you. I don't know if I should flag this as solved right now.
 
try it out. if it it does not occur after some time, please mark it as solved
 
  • Like
Reactions: bitflow

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!