Hello all,
i just noticed, that one of my Cluster Nodes shows following log every 1-2 minutes:
FYI: The Cluster works and Ceph is showing all OK
Following file exists on the node:
The file also has the same permissions as on the other nodes.
Can someone help me find the Problem?
Thanks in advance.
i just noticed, that one of my Cluster Nodes shows following log every 1-2 minutes:
FYI: The Cluster works and Ceph is showing all OK
Code:
Feb 08 01:18:52 pve-2 pvedaemon[3570]: <root@pam> successful auth for user 'monitoring@pve'
Feb 08 01:18:54 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:36.179742Z_a5b60851-068b-4d68-8a79-c498921ca432 as client.crash.pve-2 failed: 2024-02-08T01:18:54.143+0100 7f5e864366c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.147+0100 7f5e864366c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.147+0100 7f5e864366c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.147+0100 7f5e864366c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.147+0100 7f5e864366c0 -1 monclient: keyring not found
Feb 08 01:18:54 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:54 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:36.179742Z_a5b60851-068b-4d68-8a79-c498921ca432 as client.crash failed: 2024-02-08T01:18:54.371+0100 7f7385aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.379+0100 7f7385aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.383+0100 7f7385aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.383+0100 7f7385aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.383+0100 7f7385aa76c0 -1 monclient: keyring not found
Feb 08 01:18:54 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:54 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:36.179742Z_a5b60851-068b-4d68-8a79-c498921ca432 as client.admin failed: 2024-02-08T01:18:54.707+0100 7fd41b1ed6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.715+0100 7fd41b1ed6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.719+0100 7fd41b1ed6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.719+0100 7fd41b1ed6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:54 pve-2 ceph-crash[2062]: 2024-02-08T01:18:54.719+0100 7fd41b1ed6c0 -1 monclient: keyring not found
Feb 08 01:18:54 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:55 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:29.669361Z_1185fd7a-d051-4b5d-badf-bba3179e4d74 as client.crash.pve-2 failed: 2024-02-08T01:18:55.027+0100 7f34101fb6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.035+0100 7f34101fb6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.035+0100 7f34101fb6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.035+0100 7f34101fb6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.035+0100 7f34101fb6c0 -1 monclient: keyring not found
Feb 08 01:18:55 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:55 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:29.669361Z_1185fd7a-d051-4b5d-badf-bba3179e4d74 as client.crash failed: 2024-02-08T01:18:55.359+0100 7f83d6aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.371+0100 7f83d6aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.371+0100 7f83d6aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.371+0100 7f83d6aa76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.371+0100 7f83d6aa76c0 -1 monclient: keyring not found
Feb 08 01:18:55 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:55 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:29.669361Z_1185fd7a-d051-4b5d-badf-bba3179e4d74 as client.admin failed: 2024-02-08T01:18:55.699+0100 7fc752c846c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.707+0100 7fc752c846c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.711+0100 7fc752c846c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.711+0100 7fc752c846c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:55 pve-2 ceph-crash[2062]: 2024-02-08T01:18:55.711+0100 7fc752c846c0 -1 monclient: keyring not found
Feb 08 01:18:55 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:56 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:30.031210Z_5e08968a-0210-42bd-93e1-bffc839edfbc as client.crash.pve-2 failed: 2024-02-08T01:18:56.039+0100 7f707d9726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.047+0100 7f707d9726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.047+0100 7f707d9726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.047+0100 7f707d9726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.047+0100 7f707d9726c0 -1 monclient: keyring not found
Feb 08 01:18:56 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:56 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:30.031210Z_5e08968a-0210-42bd-93e1-bffc839edfbc as client.crash failed: 2024-02-08T01:18:56.375+0100 7f53fb4a76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.383+0100 7f53fb4a76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.387+0100 7f53fb4a76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.387+0100 7f53fb4a76c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.387+0100 7f53fb4a76c0 -1 monclient: keyring not found
Feb 08 01:18:56 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:56 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2024-02-05T09:58:30.031210Z_5e08968a-0210-42bd-93e1-bffc839edfbc as client.admin failed: 2024-02-08T01:18:56.711+0100 7f4e7ab726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.719+0100 7f4e7ab726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.723+0100 7f4e7ab726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.723+0100 7f4e7ab726c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:56 pve-2 ceph-crash[2062]: 2024-02-08T01:18:56.723+0100 7f4e7ab726c0 -1 monclient: keyring not found
Feb 08 01:18:56 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:57 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2023-11-26T00:40:29.504825Z_f1e52509-1dcd-4d0e-8088-0b92c3805269 as client.crash.pve-2 failed: 2024-02-08T01:18:57.007+0100 7fbcd90106c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.019+0100 7fbcd90106c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.019+0100 7fbcd90106c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.019+0100 7fbcd90106c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.pve-2.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.019+0100 7fbcd90106c0 -1 monclient: keyring not found
Feb 08 01:18:57 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:57 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2023-11-26T00:40:29.504825Z_f1e52509-1dcd-4d0e-8088-0b92c3805269 as client.crash failed: 2024-02-08T01:18:57.311+0100 7f1b1523d6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.319+0100 7f1b1523d6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.323+0100 7f1b1523d6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.323+0100 7f1b1523d6c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.crash.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.323+0100 7f1b1523d6c0 -1 monclient: keyring not found
Feb 08 01:18:57 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)
Feb 08 01:18:57 pve-2 ceph-crash[2062]: WARNING:ceph-crash:post /var/lib/ceph/crash/2023-11-26T00:40:29.504825Z_f1e52509-1dcd-4d0e-8088-0b92c3805269 as client.admin failed: 2024-02-08T01:18:57.651+0100 7f5f748c46c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.659+0100 7f5f748c46c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.659+0100 7f5f748c46c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.663+0100 7f5f748c46c0 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.admin.keyring: (13) Permission denied
Feb 08 01:18:57 pve-2 ceph-crash[2062]: 2024-02-08T01:18:57.663+0100 7f5f748c46c0 -1 monclient: keyring not found
Feb 08 01:18:57 pve-2 ceph-crash[2062]: [errno 13] RADOS permission denied (error connecting to the cluster)Following file exists on the node:
/etc/pve/priv/ceph.client.admin.keyringThe file also has the same permissions as on the other nodes.
Can someone help me find the Problem?
Thanks in advance.
Last edited:
