Search results

Thank you. That fits very nicely in the "why didn't I think of that" category. Worked as expected.

I have a 13 node cluster using HA. What are the best practices for setting the max_restart and max_relocate values? As it stands right now, for VMs that can run on any node, I've simply picked a restart value of 4, and a max_relocate of 10. My thinking is that the HA service will try to...

So I have a 13 node cluster running a about 40 VMs. There a number of VMs scattered across the nodes that are in the HA state of error, but the VM iteslf is actually running. After moving the first VM from error->disabled->started, I realized that it was shutting down and restarting the VMs...

Everything is stable right now, but it still doesn't feel as fast as it should. All the benchmarks show that the speed is there, but in practice the transfers as I move my images to the ceph storage still don't feel as fast as they should. One of the problems I had was playing around with MTU...

Ah. No, I don't have that set. Not using storage for containers.

Moving the disk is also tied to CPU usage since it appears to be using the qemu-img command to convert the image to the ceph storage. That also seems to be limiting the speed when moving a disk from local to ceph. I don't know how to answer your first question. I've installed ceph from the...

Yes, that is interesting. Even though iperf shows 10G transfer bi-directional, using scp the 30Gig file transfers about the same 35MB/s rates. I'll look into what may be happening and report back.

3. I'm copying a 30G image right now. It's already been a few minutes and the GUI shows 35MB/s.

It does seem faster, but not 10G faster.

Yes, all servers are synchronized to the proper time. I did find that when I deleted all my monitors except on the three storage nodes, the errors went away. I think there is something not happening (or happening incorrectly) when I created the extra monitors on other nodes.

Followup... When trying to do some rbd commands, I get this sometimes: 7f19cf41d700 0 -- 192.168.201.236:0/4227972101 >> 192.168.201.245:6800/11670 conn(0x7f19b8017eb0 :-1 s=STATE_CONNECTING_WAIT_CONNECT_REPLY_AUTH pgs=0 cs=0 l=1).handle_connect_reply connect got BADAUTHORIZER If I abort the...

I am running PVE5.0 with Ceph Luminous from the test repository. I have a 13 node Ceph & PVE cluster with three storage nodes, each storage node has four 2-TB osd with bluestore backing. and a SSD drive for the journal (block.db) on partitions. The nodes are communicating over a 10Gb network...

Yes, there is a RAID controller on all three hosts, which is why the SSD shows up as a regular HD. The RAID controller says it is optimizing things for the SSD behind the scenes. There is a SSD on sdb, and the OSDs are sdc, sdd, sde, sdf. When I configured an OSD, I selected the data drive...

Thanks for all the help. Two more questions now that I have the third storage node up and added to the cluster and healthy: When I created the OSDs with Bluestore as the backing, it only created 1GB block.db partitions on my SSD, probably because the system didn't correctly detect the SSD and...

Perhaps I wasn't as specific as I could be. The ceph cluster is using the same hardware as my 13 node PVE 5 cluster. I currently have only two storage nodes (which are also PVE nodes), but I will be adding new hard drives to one of the PVE nodes to create a third ceph storage node. The...

The answer to your first question is yes. You can disable local storage in the storage configuration and it will no longer show in the GUI on the left side in any of the views. Since I use local storage currently, I can't confirm that. In the worst case, you can remove all of the object types...

I added another monitor per your suggestion...but since I only have two nodes operational right now, I don't see the point of having three replicas yet. You say it's risky, but when I used a NAS server, that was a single point of failure...so what I asked is whether or not with the two replicas...

After a year of using freeNAS with LVM over iSCSI with 30 nodes and almost 100 VMs, that solution finally revealed its shortcomings. As a stopgap to keep the VMs running, I moved the VM images back onto each node's local hard drives. I'm now looking to setup a three node ceph cluster, running...

Good job. I've been having a similar problem. I'm now moving to ceph storage because I didn't know exactly what the source of the problem was.

My understanding is that you move the "requested state" to "disabled" for the VM in HA resources, then back to "start". That should clear the error.