Search results

Do not use HDD only with Ceph.

Are these two separate CephFS volumes? What does "ceph fs status" show? If that's the case you need to specify the fs name with the fs= option to the mount command.

Ceph is not able to run dual stack. You have to either use IPv4 or IPv6 on both networks.

Failure donain host will not allow two copies on the same host. All PGs of the pool will be degraded in such a situation as Ceph is not able to find a location for the fourth copy.

No, the size of the pool defines the number of copies it creates (when being replicated, not erasure coded). It has nothing to do with the number of nodes. A size=3 pool will distribute the three copies over four nodes randomly. If an OSD is down the missing third copy will be recreated from...

Your datastore01 pool has a size of 4. It stores four copies of each object, hence its usable capacity is only 25% of the total capacity.

This is Ethernet and not Fibre-Channel. You cannot have two separate interfaces in the same VLAN. Create a stack out of the two switches (or MLAG or whatever the vendor calls it) and then a link aggregation group for each Ceph node with one port from each switch. On the Ceph nodes create a...

This is for the rule, it is not the size of the pool.

Why are there 8TB data stored in device_health_metrics? Do you store RBD data in this pool? You should create a separate pool for RBD data. https://pve.proxmox.com/wiki/Ceph_Pacific_to_Quincy#Guest_images_are_stored_on_pool_device_health_metrics

Please post the output of ceph df.

Are there other pools using the same OSDs?

Can you ping the IP address? What does "curl -v http://ip_address" show?

This may be because this host is not part of the Proxmox cluster.

/etc/ceph/ceph.conf is a symlink to /etc/pve/ceph.cobf on Proxmox nodes. /etc/pve is the FUSE mounted ckustered Proxmox config database. You should check why ceph.conf is not available. BTW: do not run with an even number of MONs. Add one (less risk) or remove one (equal risk as with 4).

The target ratio of a pool is for the balancer and does not influence the total capacity calculation. You should set the nearfull_ratio from 0.85 to 0.66. ceph osd set-nearfull-ratio 0.67

The IPv4 packets come from the same MAC address as the IPv6 packets. Have you removed 94.130.55.108 from enp0s31f6?

Is that at a hosting provider that maybe blocks unknown MAC addresses? Do you see traffic when you run "tcpdump -i enp0s31f6 -ne" on the Proxmox host?

You should remove these IP addresses from the ethernet interface. Add a bridge (vmbr0) with the ethernet interface as a port and add the IPv6 address to the bridge interface. Atttach the VM to the bridge and configure the IPv4 address inside the VM on its interface.

No need to remove the packages, you can downgrade them with: apt install ceph-common=18.2.4-pve3 libsqlite3-mod-ceph=18.2.4-pve3 librados2=18.2.4-pve3 and possible any other package that is newer than reef.

You need to stop the MON and replace its store.db. Make sure that the MON runs on the same IP as in the old cluster. It's all in the Ceph documentation.