Search results

Ceph does not use a filesystem on its data drives.

For the second network (the Ceph cluster network) you do not need an outside connection and in particular no default gateway (there can only be one). Use the IP network you designated for this second network interface as Ceph cluster network in the configuration and Ceph should automatically...

Do you see any IO errors in the kernel log (/var/log/kern.log) for the device of OSD 8?

Ceph does not have an independent location to place the third copy. But inconsistent PGs mean there are copies that do not match. Read in the Ceph documentation about troubleshooting PGs.

Could be a bit error on the disk. Try to redeploy the OSD. First purge it from the cluster, zap the disk and create a new OSD on it.

This is the smallest possible Ceph cluster without any room for parallelization. Each OSD has to participate in every write request. The Samsung 870 QVO is a QLC SSD with a small "TurboWrite" cache of only a few gigabytes. As soon as this is full the write performance drops to around 160MB/s...

Dell says it's compatible with Ubuntu, so Proxmox should also run on it.

Yes.

Das wird so nicht funktionieren. Die Switche sind doch auch noch miteinander verbunden, oder wird der Proxmox-Host dann der Core-Switch? Auf die Art und Weise erzeugst Du eine Switch-Loop und legst das Netzwerk lahm.

Da sind nur die Lock-Infos, die da rübergehen, und ob die Knoten alle noch da sind. Trafficseitig vernachlässigbar.

Auf der separaten NIC ("Loop1") gibt es keinen Traffic von VMs. Der Storage-Traffic geht über SAS und der IP-Traffic geht über die andere NIC.

Not online. You have to redeploy each OSD and that means data movement. Usually this can be done on a host by host basis without losing too much redundancy.

You are using the same IP subnet (192.168.1.0/24) for two different Ethernet networks.

You cannot have an admin on Pool level that is also an admin for the user management or the storage management. These two are global functionality and not restricted to pools. You can restrict via permissions which storage entity a group of users can consume for the VMs in their pool.

Run ceph crash ls-new and then ceph crash archive-all. Other options can be listed with ceph crash --help.

Ah, makes sense. We install jq and a bunch of other tools by default on our Proxmox systems.

Grundsätzlich ja, denn da entsteht nicht viel Traffic. Für Corosync ist immer noch eine zweite unabhängige Netzwerkverbindung empfehlenswert.

Your cluster is not healthy. You only have 2 of 3 HDDs online. There are way too much PGs on the HDD OSDs, they should be reduced to around 200. Your "replicated_rule" does not use the device class and mixes SSDs and HDDs. You have to fix these issues before you can talk about performance...

Are these Enterprise class SSDs or consumer grade SSDs? The later tend to have a very small write cache and as soon as this is filled the write bandwidth drops below HDD speed.

Neat. To parse the JSON output you could use jq: pvesh get /storage/$DATASTORE_ID --output-format json-pretty | jq .disable