Recent content by franciscopaniskaseker

  1. F

    Ceph x daemons have recently crashed

    Since last Ceph update (to current 17.2.5) we noted that every node reboot will mark OSDs from that node as crashed. However they return with the server boot normally. I checked ceph and journalctl logs, and I did not find anything relevant about the daemons crashing (timeout, segfault, etc)...
  2. F

    Ceph 17.2 Quincy Available as Stable Release

    No errors in upgrade procedure: Log started: 2023-01-30 17:33:46 Selecting previously unselected package ceph-volume.^M (Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading...
  3. F

    Ceph 17.2 Quincy Available as Stable Release

    Seems that some error happened, but not reported, and the ceph-volume package was missing. apt install ceph-volume ceph-volume lvm activate --all fixed the problem.
  4. F

    Ceph 17.2 Quincy Available as Stable Release

    Today we did proxmox and ceph update and after reboot 1 of 3 servers the OSD stopped to work with error: Jan 30 13:38:49 pve3 systemd[1]: Started Ceph object storage daemon osd.4. Jan 30 13:38:49 pve3 ceph-osd[18974]: 2023-01-30T13:38:49.422+0000 7ff335e73240 -1 auth: unable to find a keyring on...
  5. F

    keyring: (2) No such file or directory

    i am having same error after a reboot. did you find any solution?
  6. F

    `/var/lib/ceph/osd/ceph-<ID>/keyring` is gone.

    Happened with me today. I did not find the ceph-volume-systemd command. I have exacty same errors after reboot.
  7. F

    Using KSM in proxmox nodes, but and inside of the VM?

    KSM is an excellent feature to reduce de memory page and I use it with Proxmox and it works really well. Usually the VM does not has KSM enabled, so I am wondering if setup a KSM inside of VM can help reduce the memory usage or it will make KSM under proxmox node less efficient or it will not...
  8. F

    Losing dhcp ip

    Seems that we found the issue: We disabled ipv6 stack in sysctl.conf file in the past. I checked that networking service can not restart with 100% success, but the ipv4 config was being renewed and in the servers that I did this, they did not have the problem anymore. Talking with hetzner...
  9. F

    Losing dhcp ip

    Thank you for reply. Yes, hostname is solving. About journalctl, I found some stuffs, but I am not sure how relevant they are. for example: Sep 27 15:21:26 gateway.domain.com pmgmirror[1232]: database sync 'gateway3' failed - DBI...
  10. F

    Losing dhcp ip

    I have 4 servers running PMG (debian 11.5 and last PMG stable version). The IP is configured through DHCP from the datacenter. Everyday all VMs are losing the DHCP, and they lose at the same time (they stop to talk with each other and then they lose the external network comunication). When I...
  11. F

    Which events can trigger automatic pve nodes reboot?

    Today we are seeing this again. We shutdown one node, and the others too reboot also. We tried to add qdevice to help with the quorum, but is not working; And we are seeing very weird informations... root@pve3 ~ # pvecm qdevice setup X.Y.Z.W -f All nodes must be online! Node pve1 is offline...
  12. F

    Which events can trigger automatic pve nodes reboot?

    less than 20 minutes. I will check in next reboot. Votequorum information ---------------------- Expected votes: 3 Highest expected: 3 Total votes: 3 Quorum: 2 I executed now. Is not expected quorum 3?
  13. F

    Which events can trigger automatic pve nodes reboot?

    If I have 3 nodes and I execute reboot in just one node, is it possible to that node be temporary removed from proxmox, causing HA quorum problem? This can explain why the other 2 servers rebooted automatically.
  14. F

    Which events can trigger automatic pve nodes reboot?

    Reading the topics: https://forum.proxmox.com/threads/reboot-in-cluster-gui-is-restarting-all-nodes.77053/ https://forum.proxmox.com/threads/all-servers-from-cluster-reboot-after-one-server-reboots.37278/ seems possible.
  15. F

    Which events can trigger automatic pve nodes reboot?

    Problem: My cluster rebooted two bare metals without being asked to do. What I was doing: I was updating the proxmox/debian packages (proxmox 6.x and debian 10.11). I have one cluster with 3 nodes. I will call them pve1, pve2 and pve3. I started to move pve3 Vms to pve2 and pve1 (taking care...