Search results

Are you sure that you set min_size for this pool to 1? Please show 'ceph health detail' when cluster is in health_warn state.

Try this: echo 2 > /sys/kernel/mm/ksm/run

http://docs.ceph.com/docs/master/rbd/qemu-rbd/#qemu-cache-options

If your pool has size = 3, then each osd has (1024 * 3 / 12) = 256 placegroups. Now you'll have to: - add a new node with 4 osds (or add 4 osds to existing nodes), so there will be (1024 * 3 / 16) = 192 pg per osd (and this is the best way); - change variable 'mon pg warn max per osd' to some...

Please show your crush map (from Ceph->Configuration).

To update pve-kernel you need to restart a node.

Explanation and solution: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-December/023223.html

http://ceph.com/community/new-luminous-pg-overdose-protection/

Recovery is in progress. Wail until it complete. I wonder what is the primary cause of this failure. Maybe you didn't wait for HEALTH_OK between every step of upgrade? Or upgrade with noout set? When status become HEALTH_ERR, after reboot of last node?

This happens when you use wrong command to remove osd - ceph osd rm osd.16 instead of ceph osd rm 16 Have you tried: - reboot a node containing osd.16 (with noout flag)? - set osd.16 as lost?

So, you install PVE 5.1 on the new hardware and move osd disks from old one to new? IMO it will not work that way.

No :)

Have you tried this: http://docs.ceph.com/docs/master/rados/troubleshooting/troubleshooting-pg/ ?

Cluster is in a normal recovery state. So where is the bottleneck? Try to find it with the atop utility. Disks are SSDs or spinners? Did you have a problem with flapping osds?

Your ceph network is 1Gbps? You don't need to have more than 3 mons. If you need more storage space, you should add a node with OSDs only (without mon). It is no good for recovery/backfill process.

Please share your ceph.conf. Why 4 mons?

Add default gateway to Proxmox: #ip route add default via 192.168.1.1

You don't need to delete VMs, nor even stop them.

If you don't know what the igmp snooping exactly do, you don't need it.

You should disable IGMP snooping on both switches.