Search results

found a workaround that fixed it, updated on first post

i prefer not to touch working storage's (the cluster currently used our company) systemctl status pvestatd shows enabled and running

Ceph working on all other nodes, and the ladt mount(docs) in this picture is nfs based , hosted on qnap and it is accessible via cmd

no change, all networks are marked as down, going inside network mounted storage (that marked as down) via cmd is correct, all file are there and accessible

yes df is working, and return correct values, and mount|grep nfs works as well

I have a cluster of around 15 nodes. Today i successfully upgraded pbs from 3.0 to 3.3.2. (reboot and full test of backup\restore worked ) Then i Tried to upgrade a single node (from the GUI). all went normal and system worked after upgrade. i Initiated a reboot, to make sure everything...

i came across it , did not help. But i think i found a reason, still not sure if it caused it. all Proxmox nodes have chrony services and should sync by it. i have two more dedicated servers DC, DC backup and the backup had a time issue, after it fixed most of the issues resolved But i dont...

I have a cluster of around 20 nodes, Proxmox 8.0.4 +ceph (on 4 of them) all have the same configuration of chronyd. and most of the time everything works. recently (in the past month) i started to have issues with time sync. usually it comes after a power failure, some servers are out of...

does a benchmark exist with more then 3 nodes for example in the area of 10 nodes? does the throughput scales accordingly for multi client usage?

Update - full shut down of all ceph nodes solved the issue (one by one did not help)

There is another change i noticed today: pgs scrub issue. *till now the systems are running and responsive but i dont think is is healy

ceph osd pool ls detail pool 2 'ceph-lxc' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode on last_change 227165 lfor 0/136355/136651 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd removed_snaps_queue...

ceph -s cluster: id: 8ebca482-f985-4e74-9ff8-35e03a1af15e health: HEALTH_WARN Degraded data redundancy: 1608/62722158 objects degraded (0.003%), 28 pgs degraded, 22 pgs undersized services: mon: 3 daemons, quorum pve-srv2,pve-srv3,pve-srv4 (age 2d) mgr...

flow: 1 servers had reboot due to power maintenance, 2 (after the reboot) i noticed one server had bad clock sync - fixing the issue and another reboot solved it) the 3. after time sync fixed cluster started to load and rebalance, 4 it hang at error state (data looks ok and everything stable and...

We have a setup of around 30 servers, 4 of them with ceph storage, Unfortunately we have many power outages in our building and the backup battery does not last for long periods , casing entire cluster crash, (server, switches, storages) Most of the time the entire cluster turn up when the...

it requires reboot to reload the new config for the kernel

i am trying to reboot lxc host as part of an ansible script , however i cannot make it work using ansible, default reboot module did not work: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/reboot_module.html it printed Socket exception: Connection reset by peer (104) and...

unfortunately i rolled back to kernel 5.15 on all hosts with vms (after rollback no issues at all) we use the servers in our production so i cannot risk another downtime. the affect is only for VM.not for lxc

settings mitgation off did not solve the issue, just reduce the occurrence due to more efficient kernel , i have around 10 Ubuntu vms that the error occurs repeatedly under load (while running all nodes at 70% cpu capacity ) in less then an 1 hour i had the error at least on one of the nodes...

Setting mitigation off reduced the amount of the errors (still the bigger the load the more errors), but going back to kernel 5.15 removed the issue entirely