[SOLVED] CEPH Security Information Email

[Tmanok]

pve4.domain.ca : Aug 7 00:08:27 : ceph : a password is required ; PWD=/ ; USER=root ; COMMAND=nvme ct500mx500ssd1 smart-log-add --json /dev/sde

This error was sent to my email four times (one for each of my OSDs) consecutively and it occurred approximately five hours after upgrading from Ceph Octopus to Ceph Pacific. I'm not sure what to make of it, I just upgraded to PVE 7.0 before the ceph upgrade. This only affected my PVE4 node.

Can somebody help identify what ceph is asking me for? Is it asking for the root user's password? Why would it require that?

Thanks!

 

[Tmanok]

This happened again this morning at four different times... Please if someone can help, these are coming from PVE.

 

[Tmanok]

After reading the manual and further googling I found that a PVE 7.0 Beta user reported this same issue to the dev mailing list, the suggestion is to check if /etc/sudoers.d/ceph-osd-smartctl exists. I'll be checking shortly and reporting back here.

https://www.mail-archive.com/pve-devel@lists.proxmox.com/msg04739.html

 

[Tmanok]

Ok so the file does exist and is populated:

root@pve4:~# ls /etc/sudoers.d/ceph-osd-smartctl
/etc/sudoers.d/ceph-osd-smartctl
root@pve4:~# cat /etc/sudoers.d/ceph-osd-smartctl
## allow ceph-osd (which runs as user ceph) to collect device health metrics


ceph ALL=NOPASSWD: /usr/sbin/smartctl -x --json=o /dev/*
ceph ALL=NOPASSWD: /usr/sbin/nvme * smart-log-add --json /dev/*

The same file contents from another node without this issue are identical:

root@pve1:~# ls /etc/sudoers.d/
ceph-osd-smartctl  README  zfs
root@pve1:~# cat /etc/sudoers.d/ceph-osd-smartctl
## allow ceph-osd (which runs as user ceph) to collect device health metrics


ceph ALL=NOPASSWD: /usr/sbin/smartctl -x --json=o /dev/*
ceph ALL=NOPASSWD: /usr/sbin/nvme * smart-log-add --json /dev/*

Additionally the file permissions on the troubled node are:

-r--r----- 1 root root 199 Jul 14 02:46 /etc/sudoers.d/ceph-osd-smartctl

and the node without issues:

-r--r----- 1 root root 199 Jul 14 02:46 /etc/sudoers.d/ceph-osd-smartctl

I've also checked that my ceph osds are not run under some different random username, all my nodes use "ceph" as their user.

So I'm not sure what else to do if ceph-osd is in the sudoers file for the two commands that it needs to perform.

 

[fiona]

Hi,
maybe also read the other replies to the mailing list thread you linked to

In any case, the solution is to install the nvme-cli package, which Proxmox VE will also do for new installations.

 

[Tmanok]

Hi Fabian!

Oh dear I only read up until the typo, I didn't realize there were further replies. Installing nvme-cli next, thank you!

 

[Tmanok]

Hey Fabian,

Package installed, but not just on PVE4, on all my nodes. So unless PVE4 sees its SATA SSDs as NVMe while my other nodes do not, it would be odd that the missing package would be responsible. Only time will tell, these seem to be sent at weird intervals but within the next day if I don't get anymore notifications then all should be well.

Also that mail thread contained some very useful information, I did not know about CEPH device metrics!
Thanks,