Ceph problem when master node is out

symmcom

Well-Known Member
Oct 28, 2012
1,075
25
48
Calgary, Canada
www.symmcom.com
Wasim

i make all the tests via oracle virtual box
I too setup my very first Ceph cluster on Virtual Environment. So i know it works in that environment.

From all the info you have provided, it is apparent to me that there something going on in your cluster installation. It is not installing/configuring as it should. Weight should no be 0, storage.cfg should does not have entry as i should.

The only thing i can suggest now is to let one of us remote access to your computer to really see what is going on. I am setup with Teamviewer. You can download Teamviewer QuickSupport from www.teamviewer.com then email me the PartnerID and password given by Teamviewer after installation.

From all the posting in this thread i dont see a valid reason why it is not working for you.
 

udo

Well-Known Member
Apr 22, 2009
5,849
159
63
Ahrensburg; Germany
I too setup my very first Ceph cluster on Virtual Environment. So i know it works in that environment.

From all the info you have provided, it is apparent to me that there something going on in your cluster installation. It is not installing/configuring as it should. Weight should no be 0, storage.cfg should does not have entry as i should.

The only thing i can suggest now is to let one of us remote access to your computer to really see what is going on. I am setup with Teamviewer. You can download Teamviewer QuickSupport from www.teamviewer.com then email me the PartnerID and password given by Teamviewer after installation.

From all the posting in this thread i dont see a valid reason why it is not working for you.
OK Wasim,
your part ;-)

Udo
 

symmcom

Well-Known Member
Oct 28, 2012
1,075
25
48
Calgary, Canada
www.symmcom.com
@Konstantinos
Not sure if you sent me that email with Teamviewer ID or not but i did not receive anything. If you have not, send it to wahmed@symmcom.com.



OK Wasim,
your part ;-)
Udo
@Udo
If i cannot figure out heads or tails of it i will hand it to you. :) You are next in line.
 

symmcom

Well-Known Member
Oct 28, 2012
1,075
25
48
Calgary, Canada
www.symmcom.com
@Udo,
As we suspected the weight of OSDs were the only cause of this issue. For reason unknown all 12 OSDs had weight of 0. So no PGs were created even though it showed all OSDs up and in. all 256 PGs were stuck and degraded. As soon as i reweighted them all PGs were created and shutting down node 1 did not effect the Ceph cluster at all.

I am suspecting it was the combination of Virtual Box and small 8GB virtual disk images for OSDs caused weight of 0. I manually set the weights to 0.006. Konstantinos installed same setup several times following Proxmox wiki to the letter. But weight was 0 every time without any manual intervention.

But.... it is all working now.
 

udo

Well-Known Member
Apr 22, 2009
5,849
159
63
Ahrensburg; Germany
@Udo,
As we suspected the weight of OSDs were the only cause of this issue. For reason unknown all 12 OSDs had weight of 0. So no PGs were created even though it showed all OSDs up and in. all 256 PGs were stuck and degraded. As soon as i reweighted them all PGs were created and shutting down node 1 did not effect the Ceph cluster at all.

I am suspecting it was the combination of Virtual Box and small 8GB virtual disk images for OSDs caused weight of 0. I manually set the weights to 0.006. Konstantinos installed same setup several times following Proxmox wiki to the letter. But weight was 0 every time without any manual intervention.

But.... it is all working now.
Hi Wasim,
we all have more time, if they read the posts: http://forum.proxmox.com/threads/20700-Ceph-problem-when-master-node-is-out?p=105666#post105666

nevertheless - yes 8GB for an disk is to less. you need 10GB to get an weight of 0.01!

Udo
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!