Recent content by finish06

Can you please explain how you have your ceph cluster set up? It is unconventional to have all SSDs on one host and all HDDs on another host.

Check the OSD ratios. Is one OSD holding more data compared to the others? This can be seen in the Ceph > OSD page, Used (%). If one of the OSDs is holding a higher percentage compared to others, you can manually set the reweigh via the command line to a lower value to help force a...

Old thread, but great advice. Thank you @aaron. This was my issue also. Could this be added to the Proxmox Wiki? I searched the Ceph documentation and didn't see mention of this issue.

I would only recommend having multiple pools if the duplication is different or if the drive type is different, e.g. HDD & SSD. If you follow this path, you must set up different rules with different classes for drive type. Again, this is only needed if you have different drive types. If all...

Look into erasure coding: https://docs.ceph.com/docs/master/rados/operations/erasure-code/. I use it on my cephFS implementation on a 3 node, 4 OSDs per node cluster. It works well. I would not use erasure coding for VM storage.

I use ceph bluestore OSDs... if you do also, review this reference doc: http://docs.ceph.com/docs/mimic/rados/configuration/bluestore-config-ref/ TLDR: set the maximum memory usage per each OSD in ceph.conf. I have done this without any issues. [global] other settings...

Good news... I read up on the Ceph docs and found the following: A pool can then be changed to use the new rule with: ceph osd pool set <pool-name> crush_rule <rule-name> AKA - Pools can have NEW rule sets applied! http://docs.ceph.com/docs/mimic/rados/operations/crush-map/

Please clarify I understand your problem correctly: the 'general' pool is storing information on the SSDs? This information is dependent upon my understanding above being correct. Just create a new temporary pool and empty everything off of the current problematic pool. Then delete the...

I think it will require a completely new pool that you migrate over too, however I am not 100% positive.

Ceph does use ALL OSDs for any pool that does not have a drive type limitation. Your theory is likely valid. Also, FYI the Total column is the amount of storage data being used. Not the total availability of the pool.

Good information. This got it working for me. Thanks!

I am still experiencing this error... must manually restart networking.service upon reboot for OVS network to start up. Does anyone know anything about this bug? Thanks!

Based upon my knowledge, however limited, if using ceph or glusterFS, you can definitely set up HA as long as you have duplicates of the data, e.g. set ceph duplication level to 3. Also, I would suspect you would be able to do ceph on a 3 node cluster with 1 SSD journal device (I would...

I would definitely recommend considering ceph. However, I would definitely want to have a subscription for the customer d/t the upcoming changes in Proxmox with the recent Hammer to Jewel upgrade. Also, "pveceph install does not run correctly due to changes on the ceph end. It is easy to...

Constant error in syslog for host ... pvedaemon (4 to 5 digit number) got timeout. Here is my package versions... proxmox-ve: 4.2-54 (running kernel: 4.4.10-1-pve) pve-manager: 4.2-15 (running version: 4.2-15/6669ad2c) pve-kernel-4.4.6-1-pve: 4.4.6-48 pve-kernel-4.4.10-1-pve: 4.4.10-54 lvm2...