mailcow-dockerized icon indicating copy to clipboard operation
mailcow-dockerized copied to clipboard

Feature Request: DMARC report parser

Open normanu opened this issue 7 years ago • 33 comments

It would be nice to have a parser automaticly readout the incoming reports mailbox and work it into a report like at

https://dmarcian.com/dmarc-xml/

normanu avatar Apr 30 '18 14:04 normanu

We could indeed add something like that. Not much effort. We would also need to add it to our DNS check.

We could use something like dmarc-reports@MAILCOW_HOSTNAME. A MX record does not need to exist, the A/AAAA record will be used as fallback. We can then parse it through a script and, it is a valid XML report, write it to our database.

andryyy avatar Apr 30 '18 14:04 andryyy

I see http://dmarc.postmarkapp.com/ does it for free, but it might be better to keep it on the own server?

normanu avatar Apr 30 '18 15:04 normanu

http://dmarc.postmarkapp.com/ only supports a single domain per account. https://www.dmarcanalyzer.com also has a free tier, but it is quite limited. Building something into Mailcow would be nice, but doesn't need to be a super high priority -- one can deploy DMARC just fine without monitoring these reports.

mkuron avatar Apr 30 '18 15:04 mkuron

I just wanted to drop my opinion here too, I would love to have a dmarc analyzer build into mailcow. I currently use http://dmarc.postmarkapp.com/. But using a build in solution would be really great also in terms of privacy.

ntimo avatar May 01 '18 09:05 ntimo

Solved that for my personal setup with these: https://github.com/techsneeze/dmarcts-report-parser https://github.com/techsneeze/dmarcts-report-viewer Mailcow-official would be nice :)

ledodev avatar May 05 '18 08:05 ledodev

Would it be possible to implement this into Mailcow? https://github.com/techsneeze/dmarcts-report-parser https://github.com/techsneeze/dmarcts-report-viewer

ntimo avatar Jun 08 '18 17:06 ntimo

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Oct 13 '18 21:10 stale[bot]

Any updates on this @andryyy ? DMARC and other mail security features (like SRS) are increasingly important these days. It can be integrated individually, however, such important things should be part of the Mailcow core.

It somehow feels these things are not getting the priority it deserves.

SomeGeek avatar Oct 18 '18 23:10 SomeGeek

A DMARC parser will gain you security? There are a million services to parse them with nice reports etc. DMARC already is part of mailcow (Rspamd).

I don’t understand how a parser deserves higher priority. :-/ It’s just a nice to have, I guess.

andryyy avatar Oct 19 '18 03:10 andryyy

In my opinion DMARC doesn't make sense when you don't have insights in it. Sure, it can be done right now, but it should be a core feature.

SomeGeek avatar Oct 19 '18 17:10 SomeGeek

We already use DMARC, we just don't have an analyzer. This is not a core feature. It is a nice-to-have. Rspamd uses DMARC, it is not that we ignore it.

andryyy avatar Oct 19 '18 18:10 andryyy

@SomeGeek @ntimo I have made a PR to integrate @ntimo 's request to add in the reporter.

https://github.com/mailcow/mailcow-dockerized/pull/2065

Clete2 avatar Dec 03 '18 19:12 Clete2

it would be very helpful to see progress here.

Adorfer avatar Jan 25 '19 18:01 Adorfer

https://hub.docker.com/r/gutmensch/dmarc-report Sounds pretty good.

vain90 avatar May 06 '19 12:05 vain90

As i read from #2065 the dmarc report is welcome, but a docker container for that is overload. So it seems that the goal is to integrate it in the Interface (via PHP).

develth avatar May 23 '19 09:05 develth

If somebody interested I created dedicated simple docker-compose for this stuff. It uses parsedmarc as parser and elasticsearch as storage and kibana as visualizer. On top it use nginx for basic auth and ssl. Maybe it will help somebody: https://github.com/dragoangel/parsedmarc-dockerized

dragoangel avatar Nov 12 '19 14:11 dragoangel

Adding 2cents.

In today’s email world, you need a proper feedback loop. I understand the fine line between must have and nice to have, but w/o a proper feedback loop it’s becoming harder and harder to make it i to a persons inbox w/o first being labeled as spam.

borrelan avatar Aug 28 '21 15:08 borrelan

A few solutions were linked already. Did anyone configure a working configuration already?

While the question of a default component is not agreed, this could be documented in the third party tools area: https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic/

ThomDietrich avatar Oct 01 '21 16:10 ThomDietrich

A few solutions were linked already. Did anyone configure a working configuration already?

While the question of a default component is not agreed, this could be documented in the third party tools area: https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic/

It already documenteddocumented at https://mailcow.github.io/mailcow-dockerized-docs/prerequisite-dns/

dragoangel avatar Oct 01 '21 17:10 dragoangel

@dragoangel thanks for your answer. With "it" you refer to the usage of an external service. While I am generally not against that, for someone self-hosting a mailcow instance for the obvious reasons, this is not really the answer we are looking for.

Did anyone already include parsedmarc in their mailcow-dockerized setup? Would be nice if we could document that for broader adoption by others. Happy to contribute if no one steps forward.

ThomDietrich avatar Oct 03 '21 12:10 ThomDietrich

Why you think should be part of mailcow stack?

I refer docs, which mentioned self hosted parsedmarc solution which can be freely deployed on same host as mailcow or dedicated one. This not should be a part of mailcow docker-compose project to work - it depends on heavy ELK stack to work with. There is a plenty people who use it and I'm part of them.

dragoangel avatar Oct 03 '21 12:10 dragoangel

Let's not restart this discussion. The arguments were very clearly presented by half of the people involved in this thread.

It is totally fine if the vote is not to include this capability as a default (which I indeed agree with), but let's enable those who are interested by providing the right details to make this effortless. It's just silly that articles next to https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic give instructions for Nextcloud or Gitea, but not about a component crucial to have constant visibility on the DMARC integrity of your mail domain.

After a quick look it seems like there are two projects that offer the right stack to deploy parsedmarc:

  • https://github.com/dragoangel/parsedmarc-dockerized
  • https://github.com/patschi/parsedmarc-dockerized

The second one looks better to me. I might give it a try when it's not sunny outside ;)

ThomDietrich avatar Oct 03 '21 12:10 ThomDietrich

@ThomDietrich do you have any news about this topic? PS: It is winter outside ;)

lukaspavelka avatar Feb 07 '22 18:02 lukaspavelka

Would be nice if added @DerLinkman

vermium avatar Jan 20 '23 12:01 vermium

I didn't get to it. Nor did anyone else in this thread it seems.

Thinking about it, this is the perfect proof for why a solution needs to be documented (or potentially offered). 15 participants and 5 years later, and people are still asking about it 🤷‍♂️

ThomDietrich avatar Jan 20 '23 13:01 ThomDietrich

Any news here?

hanebuechenes avatar Jan 19 '24 15:01 hanebuechenes

The point that it documented

https://docs.mailcow.email/getstarted/prerequisite-dns/?h=dmarc#optional-dmarc-statistics

But looks like people not like to read docs :P I don't think we should do this as part of mailcow. If people want it they can have it, as selfhosted or saas solution.

@DerLinkman I think we can close this issue honestly

dragoangel avatar Jan 19 '24 16:01 dragoangel

I read the docs. Maybe you could help stitch the docs with the platform?

All I see is external independent tools that are not integrated. Unless I missed something?

Update: In 2024 DMARC is more important than ever, yet the tools to manage it are not integrated to reflect its growing importance. I guess that the voice of the minority, but I have long moved away from self hosting as it's impractical without deliverability tools these days.

borrelan avatar Jan 19 '24 16:01 borrelan

I read the docs. Maybe you could help stitch the docs with the platform?

All I see is external independent tools that are not integrated. Unless I missed something?

This dedicated topic that should not be "integrated", I don't understand why people want them to be part of mailcow. There 0 reason why they should be a part. Storing such historical data in ELK is heavy thing to add, it don't need on same host. Using SaaS at all not require you use your own server...

dragoangel avatar Jan 19 '24 16:01 dragoangel

You're not wrong, but the platform is useless without these tools to reach large email providers. I guess having your emails reach the destination is optional in your opinion?

borrelan avatar Jan 19 '24 16:01 borrelan