Notification system does not pick up root mails

[mpeter]

Hi!

I use PVE 8.3.5, and I wanted to set up the notification system so that I get notified through Ntfy when something happens.
I have made a webhook notification target and when I click test, I receive the notification in the Ntfy webapp so that part should be fine.

I have also made a new notification matcher with no match rules in the hopes that it would pick up mails sent to root. I have checked the box of the Ntfy notification target.
However if I test it by sending a mail to root with the below command, I wont get a message through Ntfy:

```
mail -s "Proxmox-Ntfy webhook teszt" root < /dev/null
```

If I open my mailbox (my account is set up for root's alias in /etc/aliases but I think that is standard) with the mail command, the mail is there (and every time I repeat it an additional one appears). root's mailbox is empty and /var/mail/ has no file for root.

The `/root/.forward` file contains this line, along with the pipe:
```
|/usr/bin/proxmox-mail-forward
```

If I watch the system journal while sending the mail, I see informational messages of postfix (postfix/pickup, postfix/cleanup, postfix/qmgr, postfix/local), but nothing else.

What may be the problem? What can I do to debug it?

 

[leesteken]

I'm seeing the same issue (on PVE 9) and found this thread while searching the forum. I also found this thread https://forum.proxmox.com/threads/system-mails-are-not-forwarded.167105/ and double checked everything accordingly but the to does not get rewritten to the e-mail address of the root user. Did you resolve this?

 

[mpeter]

Yes, I did. Actually that other post is also me

On my system the problem was that I changed the mail alias configuration because of a misunderstanding. That lead to the the mail system sending the mail to the aliased account, instead of checking /root/.forward for what to do. My final solution is here: https://forum.proxmox.com/threads/system-mails-are-not-forwarded.167105/post-777123

The forum says you are paying for Proxmox, if thats the case you should have access to some kind of a customer portal and a guaranteed response time, maybe you could check in with them: https://www.proxmox.com/en/?option=com_content&view=article&id=23

If you figure this out, consider also sending your solution here to this thread.

 

[leesteken]

The forum says you are paying for Proxmox, if thats the case you should have access to some kind of a customer portal and a guaranteed response time, maybe you could check in with them: https://www.proxmox.com/en/?option=com_content&view=article&id=23

I have the (paid) community subscription but that does not come with support tickets. I support Proxmox, not the other way around . I also sometimes need the help from the other volunteers here.

If you figure this out, consider also sending your solution here to this thread.

Don't worry, I have been posting solutions (to other people's questions, mostly though) for years.

 

[mpeter]

I have the (paid) community subscription but that does not come with support tickets. I support Proxmox, not the other way around . I also sometimes need the help from the other volunteers here.

Oh theres a community subscription! Didnt know.

Hmm, 2 things you could maybe check:
- check if the system mails are there in the root account's store
- back up the /root/.forward file and change it to run a different command for mails, maybe a custom script to debug if it has been executed and what args and envvars did it see

That should reveal whether the mails are heading to the right inbox, and whether the forward file is being taken into account.

Do you receive the system mails in any other linux account?

 

[leesteken]

All mails go to root, which has proxmox-mail-forwarder correctly set in /root/.forward and I see in the logs that proxmox-mail-forwarder. However, it does not rewrite the To to use the e-mail address that is configured for the root user.
The test mail of the Proxmox notifications system works fine.
My work-around is to add a static rewrite of the recipient in the postfix main.cf.

 

[BobhWasatch]

Try creating an /etc/aliases file (see man aliases for the format) and then run "newaliases".

ETA: I don't think .forward files ever re-wrote the To address, just the so-called "envelope sender" that it gives to the receiving server.

 

[leesteken]

Try creating an /etc/aliases file (see man aliases for the format) and then run "newaliases".

All the threads and documentation that I could find warns against adding an alias for root.

ETA: I don't think .forward files ever re-wrote the To address, just the so-called "envelope sender" that it gives to the receiving server.

I thought some threads and documentation mentioned that proxmox-mail-forwarder makes sure to send the mail to root to the mail address of the root user (as configured in Proxmox).

Maybe I'm all wrong about this and then my work-around is necessary anyway. Or maybe this changed between older Proxmox versions and the newer notification system. Maybe I'll re-investigate and try your suggestions at some point in the future.

 

[BobhWasatch]

All the threads and documentation that I could find warns against adding an alias for root.

I've been doing it wrong for 25 years then . Having a mail alias seems better than having to log in as root to read root's mail.

 

[leesteken]

I've been doing it wrong for 25 years then . Having a mail alias seems better than having to log in as root to read root's mail.

Hard to argue with actual experience. The new notification system does send mail to root to my e-mail address only mail sent via command line does not work (in recent PVE).

 

[mpeter]

I've been doing it wrong for 25 years then . Having a mail alias seems better than having to log in as root to read root's mail.

wait, do webhook notifications work for you, for system mails? initially I also had it that way because it made sense, but then I did not get any mails over my webhook system where I get notified immediately, so I needed to remove the alias to get it working

 

[leesteken]

I'm not sure what you mean by webhook notifications. The "new" notification system does work for me for notifications and I do see the mail-server settings (from the Proxmox web GUI Datacenter > Notifications > Notification Targets) used in the error message (when sending system mail to root) in the System Log.

I think I remember Proxmox forwarding (system) mails to root to the e-mail address set for root (in the Proxmox web GUI Datacenter > Permissions > Users) in the past (but then I also needed a different postfix work-around, which is now solved by Notification Targets). And I do think that that would be something someone might expect, but maybe that's where I'm going wrong?

 

[mpeter]

I was meaning the webhook type notification targets. Because until I head the alias set up, that did not work for me.



But I was also responding that to @BobhWasatch .

And I do think that that would be something someone might expect, but maybe that's where I'm going wrong?

That sounds reasonable, but I (with my relatively little experience) would also expect that the mails would also be there in root's local mailbox.

 

[leesteken]

I've setup a Notification Target (and it workd when using the Test button). But command-line mail commands don't work because the To is not rewritten to the e-mail address of the root user (as configured in the Proxmox web GUI > Datacenter > Permissions> Users). There is no local mailbox; mails just don't get delivered anywhere because the Notification Target refuses the To.

 

[mpeter]

I've setup a Notification Target (and it workd when using the Test button). But command-line mail commands don't work

Until this part, that was my experience too until I had the alias configuration.

because the To is not rewritten to the e-mail address of the root user (as configured in the Proxmox web GUI > Datacenter > Permissions> Users). There is no local mailbox; mails just don't get delivered anywhere because the Notification Target refuses the To.

Why do you think the To needs to be the email address of the root user? You could be right about that, but I dont know the reason and it may be relevant.

But also, for testing I have temporarily set an email address for the root user in Proxmox web GUI > Datacenter > Permissions> Users, and I still get root mails over my Ntfy webhook, so I dont think that an email address being set has an effect on that. Tested with a "sudo echo" without giving the password, I received the sudo email through my browser. I think all the root mail needs is to be sent to the root user's name. Unless that email address config change just does not get into effect immediately.

 

[mpeter]

You say root mails, like sudo, reports appear in root's mail storage, right? If thats the case, and /root/.forward contains the string I mentioned, the root mail should really end up in the Proxmox notification system.

Just to be extra sure, /root/.forward should have |/usr/bin/proxmox-mail-forward , without "er" at the end, and with the pipe and dirnames at the front. not any other characters, it doesnt even have a newline on my system.

You did not yet tell what kind of notification target do you want to use. Probably not relevant as the test message works, but maybe. And do you have the default-matcher notification matcher set to forward to this target? I assume you dont have 2 similarly named targets that could be mistaken for the other.
Sorry if these things sound too simple or condescending, I'm trying to think of what mistakes could I make.