ceph

Hello everyone, I have a 3-node PVE cluster that I am using for testing and learning on. On this PVE cluster, I recently created a CEPH pool and had 4 VMs residing on it. My PVE cluster was connected to my Mellanox 40GbE switch but I wanted to explore reconfiguring it for full mesh, which I...

Hey I get an error when I try to mount cephfs on one of my LXC. modprobe: FATAL: Module ceph not found in directory /lib/modules/6.14.11-3-pve failed to load ceph kernel module (1) mount error: ceph filesystem not supported by the system I have followed the step from ceph documentation...

I have just finished migrating my VMs on the cluster from a hdd pool to an ssd pool, but now that there are no vm disks or other proxmox related items left on the pool, it still is using ~7.3TiB on what i assume is orphaned data? This cluster is currently running PVE 8 with Ceph 18, but has been...

I'm running a PVE 8.4 cluster. The datastore our VMs use is an external Ceph cluster running 19.2.2. Yesterday, I found a "design flaw" in our setup. In /etc/pve/storage.cfg, there's this: rbd: pve content images krbd 0 monhost mon.ourdomain.com pool pve username blah...

I have a 3 node cluster. Has a bunch of drives, 1TB cold rust, 512 warm SataSSD And three 512 non-PLP NVMe that are Gen3. (1 Samsung SN730 and 2 Inland TN320) I know not to expect much - this is pre-prod - plan is to eventually get PLPs next year. 10Gb Emulex CNA is working very well with FRR...

We've been using Proxmox with ceph for years. Our typical cluster is... Proxmox 8.3.5 Ceph 18.2.4 10 servers 3 Enterprise SSD OSDs per server each 20Gbs between the servers 10Gbs between VMs and Ceph Public network for cephfs mounts. 1 Pool for VM deployment 2 subvolumes/pools; 1 Elastic data...

Hallo zusammen, aktuell plane ich den Aufbau eines Cluster mit 3 Nodes und Ceph als Storage mit 10G Anbindung. Darauf laufen soll verschiedenes: - div. Windows VMs (DC, FS, IIS etc.) - Linux VMs (Tailscale Router, mehrere Docker VMs, Plex, Loxberry etc. ) - LXC (Cloudflare DDNS, paperless...

I'm using PVE on three nodes, and have Ceph installed as a distributed storage on these nodes. I now want to access the Ceph storage from other servers as well, via NVMe-oF. So, what I have to do is to extend my node configuration so that the PVE nodes also act as NVME-oF targets. In principle...

First, let me contextualize our set-up: We have a 3 node cluster, where we will be using CEPH for storage hyperconvergence. We are familiarizing ourselves with CEPH and would love to have someone more experienced chiming in: All of our storage hardware are SSDs. (24x 2TB NVMe, 8 per server)...

Wir testen gerade ProxMox als VMWare Ersatz. Ich habe die Schulungen gemacht und bassierend auf dieser ohne große Probleme ein Cluster mit 3 Nodes und Ceph aufbauen können. Heute wollte ich unsere ISCSI NAS hinzufügen, aber das hat nicht so ganz geklappt. Ich konnte auf die LUN zugreifen und...

Hello everyone Quick backstory. We have 2 main programs, both of them are using SQL server on windows. We used Hyper-V, but it wasn't very fast, nor secure enogh (periodic backups). Both of these systems are crucial for our company, so the best way would be to have some sort of continuity of...

I am attempting to write an Ansible playbook that sets up my Proxmox cluster. One issue I'm running into is pveceph install doesn't have a non-interactive option. I've tried setting DEBIAN_FRONTEND=noninteractive on the task like this: - name: Install Ceph environment: DEBIAN_FRONTEND...

Hey Folks, Stashing this here as it's the only solution that worked for me and I will undoubtedly need it again. Given, $> ceph health detail ... [WRN] PG_AVAILABILITY: Reduced data availability: 1 pg inactive, 1 pg incomplete pg 7.188 is incomplete, acting [5,10,43] (reducing pool...

Hi This seems to be the best place / forum to ask question about ceph :) My understanding of ceph is the underlying storage is OSD, these are distributed between nodes. Pools are then created that sit on top of OSD's ... i think OSD's are broken into PG and PG are assigned to Pools I think ...

I'm recreating my ceph cluster due a configuration corruption. I will be reusing the same hardware. The problem is I don't have enough hard drives for two ceph clusters, but there is enough capacity. I know that you can't change the size of a erasure coded pool, but is there any way to override...

Here's what I did. When I first set up the 3-node cluster I only had 1 cable for the storage network, ie. where each host is connected 10Gb/s to a switch so they can use a shared storage scheme (Ceph - to which I'm fairly new and just learning) This week we got the extra cables ordered (because...

Hi I have built a proxmox cluster and im running ceph on there. I have another proxmox node - out side the cluster and for now don't want to connect it to the cluster but I want to share the ceph filesystem - so the rdb and a cephfs so I'm thinking i need to do something like this on the...

Hello Community, I am currently setting up our new 3-Node Proxmox Cluster, pretty new to Proxmox itself. We are Using Full-Mesh with 25Gbit/s cards for Ceph, 10Gbit/s cards for Coro/VMBR and 18 (6 per Node) SATA 6G Enterprise SSDs. Ceph performance took a bit of testing, but we are now at a...

Hi got a 6 node cluster , want to reduce the number of nodes. so lets say i have server A1 and server A4 out of a1..A6 I have ceph installed on all node I want to take the drives (OSD one) out of A1 and install them into A4 with out having to resync the entire drive. I have read its possible...

Hi guys, I have a Ceph pool that I removed by accident and then I readded it back in my proxmox cluster. I had a VM disks in there before I removed and when I added the Ceph pool back in, they're not showing up. I can see that the data is there when I take a look at the storage metrics and run...