I've been playing with ceph and after initially getting it working, I realised I had everything on the wrong network. So I deleted and removed everything, uninstalled and reinstalled ceph and began recreating the cluster using the correct network. I had to destroy / zap some drives on...
wir evaluieren derzeit eine Umstellung auf Proxmox im Enterprise-Bereich. Eine Verschlüsselung sämtlicher sensitiven Daten ist für uns unumgänglich. Ja, cold boot attacks sind uns bekannt, allerdings wollen wir die Hürden weiter erhöhen, im Falle eines physischen Diebstahls der Server...
I’m setting up ceph cluster with 3x node pve 6.2. each node got following disks
7x 6TB 7200 Enterprise SAS HDD
2x 3TB Enterprise SAS SSD
2x 400GB Enterprise SATA SSD
This setup previously used for old ceph (with filestore) cluster where it configured to use 2x 400GB SATA SSD to...
Hi, I have 5 Proxmox node + 5 osd + 5 monitor + 5 manager (all node same lineup but different HDD size and different hardware), lastly added 5., and I see this:
I have only 1 pool, for all VM:
I sometimes disable 2 Proxmox node and only 3 Proxmox node available.
Please help me understand for be clear:
Example for sure, all works fine, Ceph health OK, VMs are stored in Ceph, by default only node1 running a VMs:
node1: 1000 GB OSD (1 HDD)
node2: 1000 GB OSD (1 HDD)
node3: 1000 GB OSD (1 HDD)
node4: 500 GB OSD (1 HDD)
node5: 500 GB + 500 GB OSD (2...
During a disaster "test" (randomly removing drives), I pulled one osd and one drive that had the journals for 4 other OSDs. So I assumed this would down/out 4 possibly 5 OSDs. Upon re-inserting the drives they were given different /dev/sd[a-z] labels, now the journal disk has the SD[a-z] label...
I removed one node from the cluster using 'pvecm delnode node_name"'. (before executing this, I removed the node from monitor and manager list and shut it down)
Now, the OSDs are in down/out status and I am unable to remove it from GUI (since the node removed already).
How can I remove...
Can not create OSD for ceph.
Same error in GUI and terminal:
# pveceph osd create /dev/nvme0n1
Error: any valid prefix is expected rather than "".
command '/sbin/ip address show to '' up' failed: exit code 1
The only thing i can think of is since last time it worked was that i now have two...
I'm trying out ceph to see if it would fit our need. In my testing I tried removing all OSDs to change the disk to SSDs. The issue I'm facing is when trying to delete the last OSD I get hit with an error "no such OSD" (see attached screenshot). The command line return no OSD so it's like...
ich betreibe aktuell Proxmox 5.4-13 mit 3 Nodes
Alle nodes sind identisch ausgerüstet und beherbergen auch CEPH.
Jede Node hat aktuell 4 OSDs auf jeweils einer 1 TB SSD.
Ceph läuft 3/2
PlacementGroups habe ich auf 896 schrittweise von 256 hochgeschraubt.
Hier hat mich leider die...
I ran the updates which installed a new kernel. after the reboot the monitor did not start. Attempted to start from command line:
systemctl status firstname.lastname@example.org
● email@example.com - Ceph cluster monitor daemon
Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled...
I have a big question for the ceph cluster and I need your help or your opinion.
I installed a simple 3 nodes setup with Ceph.
In one node has 2x146 GB HW RAID 1 + 18x 600 GB 10k SAS without RAID.
(Summary we have 54 OSD device and we have to buy 3 SSD for journal)
And my big...
here I describe 1 of the 2 major issues I'm currently facing in my 8 node ceph cluster (2x MDS, 6x ODS).
The issue is that I cannot start any virtual machine KVM or container LXC; the boot process just hangs after a few seconds.
All these KVMs and LXCs have in common that their virtual...
i need help in creating osd in my partition.
in our server, we are provided with 2 nvme drive in raid-1. this is the partition:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme1n1 259:0 0 1.8T 0 disk
├─nvme1n1p1 259:1 0 511M 0 part...
after adding an OSD to Ceph it is adviseable to create a relevant entry in Crush map using a weight size depending on disk size.
ceph osd crush set osd.<id> <weight> root=default host=<hostname>
How is the weight defined depending on disk size?
Which algorithm can be...
I have created OSD on HDD w/o putting DB on faster drive.
In order to improve performance I have now a single SSD drive with 3.8TB.
How can I add DB device for every single OSD to this new SSD drive?
Which parameter in ceph.conf defines the size for the DB?
Can you confirm that...
I finished upgrade to Proxmox 6 + Ceph Nautilus on 4 node cluster.
On 2 nodes I have identified that all directories /var/lib/ceph/osd/ceph-<id>/ are empty after rebooting.
Typically the content of this directory is this:
root@ld5508:~# ls -l /var/lib/ceph/osd/ceph-70/
Hi guys I notice that OSD encryption is available under create OSD. Is there any mechanism to show which OSDs are encrypted in the UI or command line? I made some encrypted and some non-encrypted to gauge performance but unable to differentiate which are which.
ich habe 3 Node Cluster mit je 3 OSD in jedem dieser Nodes.
Meine Ceph Version ist: 14.2.1 (9257126ffb439de1652793b3e29f4c0b97a47b47) nautilus (stable).
Der Pool hat Replica 3/2 mit 128pg. Sobald ich eine VM aus einem Backup, das auf einem NFS Share liegt, herstelle zeigt der...
Dear Proxmox Team,
we have 3 Dell Servers running Proxmox 6.0. Unfortunately we encountered an issue with the setup of Ceph OSD's on individual drives. The main problem is that the given Perc H710 mini adapter does not allow "IT-Mode" / JBOD Passthrough of individual drives, and so we're stuck...