Search results

  1. P

    [SOLVED] I managed to create a ghost ceph monitor

    ls /etc/systemd/system/ceph-mon.target.wants/ ls: cannot access '/etc/systemd/system/ceph-mon.target.wants/': No such file or directory systemctl status ceph-mon@pve11.service ● ceph-mon@pve11.service - Ceph cluster monitor daemon Loaded: loaded (/lib/systemd/system/ceph-mon@.service...
  2. P

    [SOLVED] Ghost monitor in CEPH cluster

    Do you have a reference for implementing this?
  3. P

    Can't Start any CTs in cluster after performing latest updates

    I changed the pool to Ceph-CT-VM ls /etc/pve/priv/ceph Ceph-CT-VM.keyring test.keyring ceph auth get client.admin | diff -s - /etc/pve/priv/ceph/Ceph-CT-VM.keyring exported keyring for client.admin Files - and /etc/pve/priv/ceph/Ceph-CT-VM.keyring are identical Still getting error: pct fsck...
  4. P

    Ceph OSD Map

    I have a 4 node ProxMox Cluster with Ceph, 4 OSDs per node. When I run 'cat /sys/kernel/debug/ceph/*/osdmap' on each node I get the following on 3 of 4 nodes. epoch 7125 barrier 0 flags 0x588000 pool 1 'Ceph-CT-VM' type 1 size 3 min_size 2 pg_num 256 pg_num_mask 255 flags 0x1 lfor 0 read_tier...
  5. P

    Flapping Network NICs on Ceph Public Network VLAN

    I'm using untagged VLANs on Netgear fully managed switches, all of my ethernet ports are 10G, 2 Corosync Rings, Ceph Public, Ceph Cluster, and 2 for LAN network. Wouldn't all ports have an issue if it was an i40e issue? I looked a little closer at 'cat /var/log/messages | grep -i i40e' and...
  6. P

    Flapping Network NICs on Ceph Public Network VLAN

    Same port on all 4 nodes, report way longer than able to paste here. This port is used for the Ceph Public Network VLAN... lsmod | grep -i i40e i40e 385024 0 root@pve14:~# cat /var/log/messages | grep -i i40e Jan 2 06:25:54 pve14 kernel: [560724.602777] i40e 0000:81:00.2...
  7. P

    Proxmox node: can't mount NFS share

    I add NFS share from Proxmox GUI: Navigate to 'Datacenter/Storage/Add/NFS' ID: <name for share> Server: <IP address for NFS share> Export: <path to folder on NFS share> Content: <choose what is allowed to be stored> Nodes: All Enable: yes
  8. P

    [SOLVED] Ghost monitor in CEPH cluster

    I tried to destroy the Monitor and Manager on one node. I now have a ghost Monitor that is both 'no such monitor id 'pve11' (500)' and 'monitor 'pve11' already exists (500)'. The Manager was destroyed with no issue. I also created a new Monitor/Manager on my 4th node to keep the count at 3...
  9. P

    HIgh IO delay on one node of cluster

    This node running steady around 7% IO delay. Spikes to 90%. Other nodes less than 1%... Snapshot of all 4 nodes in cluster...
  10. P

    Change server network IP's/Nic's?

    https://pve.proxmox.com/wiki/Separate_Cluster_Network
  11. P

    HIgh IO delay on one node of cluster

    will do next time it happens...
  12. P

    HIgh IO delay on one node of cluster

    I'm looking for ideas on tracking down the cause of this seemingly random high IO that happens on varying nodes and lasts for 30 minutes to a few hours and then goes away. I thought this problem went away with the last large update, but it's back... The only other coincidence I see that it seems...
  13. P

    Can't Start any CTs in cluster after performing latest updates

    Yes, keyring exists, But still getting same message, Still wondering if this is a false error? proxmox-ve: 6.1-2 (running kernel: 5.3.13-1-pve)
  14. P

    PVE Crashing Starting LXC Container (5.1)

    I'm seeing same symptoms with VPN running in a VM using Ubuntu 19.10 Desktop. Causes node to go offline and then reboot. proxmox-ve: 6.1-2 (running kernel: 5.3.13-1-pve)
  15. P

    NTP does not synchronize time on nodes

    Possible Solution: https://brainwreckedtech.wordpress.com/2008/08/25/howto-run-your-own-ntp-server-when-your-isp-blocks-ports/
  16. P

    Ceph 14.2.5 - get_health_metrics reporting 1 slow ops

    ceph mgr module enable crash module 'crash' is already enabled (always-on) ceph crash stat 0 crashes recorded Dec 17 21:59:55 pve11 systemd[1]: Stopped Ceph cluster monitor daemon. Dec 17 21:59:55 pve11 systemd[1]: Started Ceph cluster monitor daemon. Dec 17 22:00:21 pve11 ceph-mon[1477954]...
  17. P

    NTP does not synchronize time on nodes

    I noticed: System clock synchronized: no maybe this will help: https://askubuntu.com/questions/1058593/how-to-sync-the-time-to-network-with-timedatectl-on-ubuntu-18-04 I also use my router for LAN NTP Server...
  18. P

    Ceph 14.2.5 - get_health_metrics reporting 1 slow ops

    Did upgrades today that included Ceph 14.2.5, Had to restart all OSDs, Monitors, and Managers. After restarting all Monitors and Managers was still getting errors every 5 seconds: Dec 17 21:59:05 pve11 ceph-mon[3925461]: 2019-12-17 21:59:05.214 7f29ff2c5700 -1 mon.pve11@0(leader) e5...