Search results

  1. D

    Cluster with 30+ nodes?

    Its seems like the more I keep adding SSD's the faster this thing keeps getting. Once i get more 3.5->2.5 converters in a slap a couple more in there and re-test without turning the software.
  2. D

    Cluster with 30+ nodes?

    Oh I didnt' answer your question. I blew out all the OSD's that I had split off the journals from and set it up so that each ssd in the cluster is in the ODS+Journal layout. Other than that I adjusted my pgp group to 400...for some reason I had it set to 0 and my pg count was set to 400. Not...
  3. D

    Cluster with 30+ nodes?

    Tests: r/w: ./fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --directory=/ceph --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75 4k read: ./fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --directory=/ceph --name=test...
  4. D

    Cluster with 30+ nodes?

    Yeah it doesn't appear that way, actually, I saw a loss in iops when I split off the journals. I think that recommendation is for HDD + SSD combos. I have like 100-200 Samsung 840 1Tb Evos laying around. I ran out of 3.5 -> 2.5 converters and had to have more ordered in. I'll have 2x 6 NVMe...
  5. D

    Cluster with 30+ nodes?

    Yeah, my 10gig gear hasn't arrived yet. The test cluster is to assess cephs ability to take a kick the balls and be able to recover. Storage performance even untuned will work just fine, even using a 1Gbps network and a LSI SAS card thats limiting my SSD throughput to 1.5Gbps.... HOWEVER! I...
  6. D

    Cluster with 30+ nodes?

    In my case, I gained nothing lol, or something so minuscule it didn't jump out as a significant level of improvements. I've seen more significant gains in IOPs and throughput by adding more disks(OSD's) than I have with messing with the journal placement. In fact just yesterday I augmented my...
  7. D

    Cluster with 30+ nodes?

    I have a pair of Juniper EX4550-32Ts being shipped in which will be running stacked. http://www.juniper.net/us/en/products-services/switching/ex-series/ex4550/ Guess we'll see, not sure how sensitive the watchdog function is to latency guess thats something I'll have to test out before all the...
  8. D

    Cluster with 30+ nodes?

    In the docs it states: "A Proxmox VE Cluster consists of several nodes (up to 32 physical nodes, probably more, dependent on network latency)." Given ample network capacity, is there a hard limit on how many nodes can participate in the cluster? I'll have about 35 hosts, I'm trying to figure...