Recent content by ph0x

Answering myself: The upstream guide has a bug. Instead of ceph-bluestore-tool --dev $DEV_PATH set-label-key --key osd_key -v $COPIED_KEYRING one has to put ceph-bluestore-tool --dev $DEV_PATH set-label-key --key osd_key -v $ROTATED_KEY et voilà, the OSDs all come up happily.

That's the way. You can write a simple script getting the device path from ceph-volume and the osd_key from /var/lib/ceph/osd/ceph-X/keyring or from ceph auth get osd.X to set the bluestore value via ceph-bluestore-tool.

The cluster is in the same state as before, there's nothing to worry about. Have you tried adding the hostname after the fqdn in /etc/hosts? That's good practice after all and might help you with this problem.

Hi there! I had to do the cephx keyring migration by hand since I have a Ceph cluster member which is not part of my PVE cluster. I followed the upstream guide from ceph.com and had a healthy cluster at some point. That is after I rebooted every machine that had locked up because I rotated the...

The upstream Ceph documentation really is sufficient. It just takes a while if you do it all by hand.

Yeah sure, I was rather hoping for a list of mandatory actions that explains what needs to be done. Just to broaden the picture before I find myself diving into the code. That would of course be the ultimate way. There's also the upstream guide which will probably help as well.

Could anyone point me to a resource which tells me what the helper script is doing internally so that I can follow those steps for an external Ceph cluster member which is neither a mon nor a PVE cluster member? The helper script says FAIL: Cannot manage 'osd.12' on 'backup', which is not a node...

I encountered this problem as well and finally found someone who dug into it. More than six years ago ... I had the most problems with VMs which have a lot of virtual nics. Setting the input policy to DROP rather than REJECT seems to resolve it.

That's what I would do, yes. If you want to reuse the OSDs there might even be a way to restore them without having the cluster go through a backfill. I didn't dig into the necessary steps so far, but that should be possible. If you deploy a completely new node I'd just create the new OSDs...

Maybe you should define eno1.1 first? Apart from that I don't think that this will work as expected. I doubt that a vlan can work with ipv4 while the native vlan is ipv6. However, that's just a guess.  

As I said one cannot really know what causes the problem. That's kind of your task to ensure that passwordless login works flawlessly because only you know your exact setup. It's definitely not an error that is very common, what can be seen by us poking in the dark. However we were able to...

There are a lot of possibilities where to look at, regarding those ssh problems. You mentioned strict ssh rules, that's where I would begin. Loosen up on the rules, try to encircle the problem. We don't know why ssh works and the above command does not, but as long as this issue isn't resolved...

If you don't need a permanent pbs this can as well be a VM.

Yeah well, there you have your problem. Fix the ssh issue and the upload will most probably work.

Won't pbs import the datastore if the .chunks folder already exists during creation of the datastore?