Search results

I had stability issues and went with iscsi to a truenas vm which also had stability issues. Turned out to be something to do with rke2 and I think it was almalinux I was using initially for some reason those vms kept dropping network long enough to bork the storage connections. K8s via...

Might be worth adding a step to enable maintenance mode https://pve.proxmox.com/wiki/High_Availability#_node_maintenance to the instructions. I forgot to do this and had one of the nodes get fenced mid-upgrade which caused some "fun" which then required me to use the OOB to get it back online...

Proxmox 7.3-6, Ui Displays : Gui shows listing images failed: (2) No such file or directory (500) on EC pool when trying to list images on an EC pool I suspect it's trying to List the EC pool instead of the corresponding metadata pool This works on a new install that's still on 7.3-3

Just installed Proxmox 7.3-3 on 3 nodes (Haven't updated yet as hopefully we're subscribing to the enterprise repo) CEPH 17.2 across 3 nodes connected by 2x10G LACP Was getting a lot of the following messages in the CEPH logs 2023-03-09T10:50:39.317446+0000 mon.localhost (mon.0) 70 : cluster...

For anyone who doesn't care about dataloss and loves to live on the edge. In the example below the pool is called "danger" ceph config set global mon_allow_pool_size_one true ceph osd pool set data_pool min_size 1 ceph osd pool set danger min_size 1 ceph osd pool set danger size 1...

Has anyone tried running Proxmox on a 12th/13th gen Intel cpu with both P and E cores? I'm guessing as these are more consumer CPU's perhaps not, but I was tempted to build something for the "homeserver". At the moment I have a cluster of very slow low power i3's to test CEPH on but I could...

Just wondering what people are doing for kubernetes PV's when running K8s/K3s in VM's on-top of a proxmox cluster. I was thinking of exposing PVE-CEPH to rook as an external cluster and doing it that way , but I'm not sure if that's likely to bite me later or if it will cause issues with...

Should have commented this earlier but wasn't really thinking about it then. Those P2's are really slow for sustained writes once you exceed the SLC cache, I was getting ~80MB/s on the one in my gaming pc (Although it's a rarely an issue in that use case as i'm rarely writing enough data in a...

Might be worth doing it overnight if you can.

From what I read the reason the Ceph devs decided to disable Trim is they believe enterprise SSD's will do it internally, but consumer ones may not. I suspect it needs to 'catch up' now that you've enabled it. You may see odd spikes when it trims but it shouldn't be constant high latency.

Probably will do initially yes.

You may need the trim support given those are consumer SSD's

I suspect this won't be needed if you are using proper enterprise SSDs but if not you may need to enable trim I believe it's ceph config set global bdev_enable_discard true But I'm taking that from my very badly written notes on what I did on my test cluster so the command may not be correct...

Some years ago I abused a 500GB Samsung evo 850 consumer SSD as a caching drive in a small server, also using it for tmp storage when writing lxc backups. It's still going after 3yrs of power on hours (it's older than that it spent a while not doing much) and it also spent some time as a games...

So I have a Proxmox machine in a co-lo with a public range on vmbr0 (It's a single server co-lo so I don't have a router/firewall I control upstream) It's firewalled using the PVE firewall, configured under the datacenter tab and I've confirmed I cannot access the Proxmox UI.etc from an...

I have a Ceph cluster on extremely under powered USFF nodes at home to learn/experience ceph (Actually pleasantly surprised with how that's performing). Do you know what the overhead will be. System has 64GB ram at the moment, it may end up with more once it gets here and I see what the memory...

Would I be crazy to run Ceph on a single node? configured for OSD level redundancy (instead of host level). Was thinking of using Ceph as I can change the resiliency settings, or replace disks with larger ones and re-balance easier than if it was just a static ZFS. (This is assuming I can put...

can also be set per OSD Supposedly shouldn't be needed with modern enterprise drives Absolutely needed with crappy consumer drives like the ones found in my homelab ;-)

It seems since yesterday there's been a CEPH update, I just upgraded from 17.2.4 to 17.2.5 and now I seem to be able to write to the EC 2,1 volumes with a OSD offline. Edit: I think it was actually the lack of Trim and the state of some of my SSD's that was the problem and it working after...

So I'm playing around with Ceph in a lab setup on some extremely low-end hardware, and I was testing EC on a 3 node cluster. I had Proxmox installed on USB sticks as originally, I didn't realise the OSD can be a partition rather than the whole disk. I started removing/replacing OSD's so I...