After 20 days testing, I have tried everything to break the ceph cluster in this configuration:
2x146GB 15k in raid1 hardware for the system
6x600GB 15k in singles disk raid0 hardware for ceph's bluestore osd
everything works very well. no problems at all and considering that I'm using non ssd...
solved:
sorry for the useless post but maybe it can be nice to know how I fixed this - in BIOS properties you have to setup QPI resources to maximum performance and in system power management setup to maximum consumption.
In bios POST you will notice 8 GT/s QPI before was 6,4.
Only difference I found was with lspci | egrep -i --color 'network|ethernet'
on node 3 I got
03:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM57810 10 Gigabit Ethernet (rev 11)
03:00.1 Ethernet controller: Broadcom Limited NetXtreme II BCM57810 10 Gigabit Ethernet (rev 11)
on...
Hi friends,
I have 3 node ceph cluster with identical server and configuration.
The ceph network is made by dual 10Gb HP 533FLR nics in a full meshed network.
with command Iperf
- from node1 to every node and from node2 to every node,
I always obtain a maximum throughput of 6,70 Gb/s...
yes you are right but test is sure difficult cause there are several circumstances that we could not fully verify and find myself in trouble afterwards. An lsi card is about 150 euro so not really expensive,.. anyway you suggest a good point using the sata interface, but I'm worried about the...
this is a big trouble cause in hba mode seems that I cannot boot from this controller. So.. before purchasing a dedicated jbod interface I want to ask if "Physical Write Cache is disabled for the Smart Arrays(there is an option for this)" in single raid0 configuration I still will have the...
yes I know about this configuration but then I will be unable to secure the proxmox system with a raid 1 config. But maybe you will suggest to me to use a zfs raid 1 for proxmox am I right?
HI to all,
I'm planning a 3 node ceph cluster using HP gen8 servers with the p420i controller,
This particulary controller don't allow me any jbod or HBA configuration, the only way to use a single disk for buiding an OSD is to setup every single disk as a RAID0 configuration. Otherwise the...
thank you my friend for your suggestions. now everything is clear, you are right in a split brain situation ceph will stop to work and you cannot write anything, after systems return to normality everithing is working fine with no data loss only a little downtime.
thank you again
thank you, yes I really want to try but how I can create a ring in virtualbox? i think it's impossible because an internal virtual network is like a virtual switch I cannot decide where to put the cables.
I mean that if a split brain happens I will probably have some data inconsistency, do you think that simply replacing the cable or shutting down one node it's enough? How is split brain situations handled in ceph after restoring the hardware failure?
yes you are right is very difficoult but maybe it can happens accidentaly during a maintenance procedure, when you have so many servers and cables in a rack things like this can happens.. but now that I know I wil be more carefull. but if a similar split brain happens which is the best procedure...
Good morning my friends,
let's say I have a 3 node proxmox/ceph cluster in a full mesh network with a dedicated dual port 10Gb in bond broadcast for ceph and other dedicated 1Gb cards for cluster and lan as showed in this tutorial.
https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server...
maybe the only way is to use same network for ceph and pve cluster comunication, but I think that this will be a bad choiche for performance and other issues.
I have a mesh network with infiniband 40G cards, but I have tested even with a simple ethernet 1GB/s switch with same results. I think this is a fencing problem. any solution here?? :(
hi my firends,
I have two networks in a 3 node proxmox 4.4 cluster.
The first one 192.168.25.0 is for proxmox cluster comunication(1Gb/s) and the second one 10.0.2.0 is for ceph network(10Gb/s).
If the 192.168.25.0 network of a node goes down, then all VMs in that node will migrate to the other...
I cannot find documentation about the best way to replace a ceph failed disk.
For the moment I simply run these commands but I'm not sure this is the right way
From CLI:
ceph osd out osd.X
service ceph stop osd.X
ceph-disk zap /dev/sdx
then From GUI:
node-->ceph-->OSD-->create OSD--> /dev/sdx...
Good afternoon everybody,
I have a big server that hosts various VMs (3 windows server 2008 and 2 debian 8), the performance of these machines seems great, so everything runs very fast. For reasons that now I can not explain to you, I need to run on this server a couple of windows 2000 pro sp4...
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.