Misleading/wrong 'Received' header

[pschlan]

We're running a web cron service (at https://cron-job.org) which is also used by many users to trigger the cron in their phpList instance. Unfortunately, phpList incorrectly adds a 'Received' header claiming that our service actually is the origin of the emails sent by the cron, for example:

Received: from node02.executor.cron-job.org [116.203.134.67] by EXAMPLE.COM with HTTP; Thu, 04 Aug 2022 15:50:01 -0300

The origin of this line seems to be in https://github.com/phpList/phplist3/blob/d9fade9a10f613d5cca481d3cca8d7675d23b30c/public_html/lists/admin/class.phplistmailer.php#L205-L225 The code adds the 'Received' header based on the IP of the client. In case of a cron, that will be the IP address of the client calling the cron job, not the IP address of the person who authored the email campaign.

The code explicitly refers http://www.spamcop.net/fom-serve/cache/369.html which states:

When a web site sends an email based on input from a web client, it should maintain the "chain of custody" for the message. This is done by including the client's IP address in the email headers in standard format.

In this case, the web client triggering the cron does not provide any input. The email which is being sent has been pre-authored and stored in a database by phpList, and does not come from the web client triggering the cron. Hence, the header is incorrect and misleading and should not be added.

In our case, it leads to numerous of wrong FBL (feedback loop) reports and abuse reports arriving at our service.

I'd suggest to either remove this completely or change the feature in a way that the IP address of the campaign author is stored when the campaign is authored/submitted from the web interface and use that one in the Received header.

[bramley]

Just to be clear, cron-job.org is requesting a URL with the secret parameter , e.g.

https://mysite.com/lists/admin/?page=processqueue&secret=XXXXXX

The Received header identifies the IP address of the person running the processqueue page in a browser. phplist doesn't add that header when processqueue is invoked from the command line, so maybe it shouldn't add it for this case either.

The code you showed is called from

        //# when the email is generated from a webpage (quite possible :-) add a "received line" to identify the origin
        if (!empty(getClientIP())) {
            $this->add_timestamp();
        }

but I don't really understand the comment

when the email is generated from a webpage (quite possible

Not sure whether that is referring to the "send a web page" feature.

This issue was raised a long time ago https://mantis.phplist.org/view.php?id=17377

and Michiel's comments seem to indicate that the header is not meant to be added when using remote queue processing.

[pschlan]

Exactly.

Maybe one idea would be to not set the header when invoked with a set secret parameter?

when the email is generated from a webpage (quite possible

Maybe it's referring to the case where the campaign send process is triggered from within the browser.

[michield]

Yes. this is done on purpose. The idea is that if someone is on their local system and processes the queue, this Received line identifies them, so that it can be tracked. It kind of fails when you use a service to run the cron, but it was decided that the tracking was more important. In fact, the cron service could decide to stop their service to you, if they consider this a breach of their terms of service, so it is beneficial to them to have this added.

phpList.com actually offers a free service to do this, which you can sign up for here: https://www.phplist.com/createaccount

We occasionally get spam reports because of this, but in general it works fine. You can set it up by using /admin/?page=hostedprocessqueuesetup

[michield]

Ah, sorry, just notice you are from the cron service, not a phpList user. Are you getting a lot of issues because of this?

[pschlan]

Ah, sorry, just notice you are from the cron service, not a phpList user. Are you getting a lot of issues because of this?

Yes, sometimes dozens to hundreds of abuse reports per day. Most of the time FBL (feedback loop) reports which are not critical (just kind of annoying), but sometimes also real abuse reports which we have to react on.

Also, these reports are absolutely not actionable for us, since the headers don't contain the cron URL which we could use to identify and ban the user.

Hence we can't do anything about it, but have to respond to abuse reports to not get our server taken down by the ISP. This is advancing to a major problem for us.

I would really appreciate if the IP added in the header could be the one of the campaign author instead of the client which triggers the queue processing. As suggested, when submitting the campaign from the phpList web interface, you could store the author's IP address in the database and use that one. I think this would also come closer to what the RFC authors had in mind for the Received headers.

[michield]

Yes, fair enough. If we were to record the IP address of the person who puts the campaign in the queue, that is a better indication of the origin of the campaign then the process that is actually running the queue.

We can work on this. It's likely to take a while to resolve your issues though, as it will depend on people upgrading to the release that has this addressed.

It would also be interesting to think of a way that you would be able to identify the client that has caused this. If you get a lot of FBLs on a client, it may be sensible to shut them down. That's beneficial to both you and phpList as the last thing we'd want is for both your service and phpList to be associated with unsolicited email (kind of maybe too late, but you can always try).

Additionally, you could bounce people off to our service, and make it solely our problem. But that's up to you, depending on the revenue it creates.

[pschlan]

Thanks!

We can work on this. It's likely to take a while to resolve your issues though, as it will depend on people upgrading to the release that has this addressed.

That's perfectly clear.

It would also be interesting to think of a way that you would be able to identify the client that has caused this.

Yes, definitely. One could think of another 2nd argument which is non-secret but unique / uuid-like and needs to be passed to the processqueue URL, which is then added to the emails as a header. In the FBL reports, we could just parse out that header and find the cron job which has the uuid in its URL...

Additionally, you could bounce people off to our service, and make it solely our problem. But that's up to you, depending on the revenue it creates.

All we could do is try to match the URL pattern and "guess" that phpList is invoked. We can't possibly check all the jobs manually...

[pschlan]

This popped up again, and we always risk shutdown/block of our servers due to this. I would really appreciate if phpList could stop injecting these false headers.