RFE: change indication/mail rule/X-header

[zenfish]

First - thanks, this is awesome.

Secondly - a small RFE... the little X that shows things is really indistinct... having even an option to make it pop more (say, changing color to red, yellow, whatever) would be really useful.

Even more greatness would be the ability to run a rule (as in preferences->rules) that would allow basically arbitrary actions (color, junk it, whatever)... I haven't looked into your code to see the magic under the hood, so not sure how possible that is (or even changing the color above, vis a vis mail APIs and such.)

Anyway, thanks again!

[zenfish]

Just reread the description... putting in an X-header would be awesome, since you already strip content out.

[apparition47]

Thanks for your input!

a small RFE... the little X that shows things is really indistinct

The ⓧ icon was intended to be quite subtle as to not look out of place with the rest of the Mail app using blues and greys (see #2 CC: @bitmanic). As an aside, I'm thinking about change the icon to maybe 👁️ in line with similar extensions I've seen.

the ability to run a rule (as in preferences->rules) that would allow basically arbitrary actions (color, junk it, whatever)

What percentage of your emails don't show the blue or grey ⓧ icon? The issue with email nowadays is that nearly every single email includes some sort of tracker. I imagine the end result being that your rules would trigger and the majority of your emails end up coloured or in the junk. So I'm not sure if this would be a helpful feature.

putting in an X-header would be awesome, since you already strip content out.

This makes sense. I'll see if I can get around to this, thanks!

[zenfish]

X-header would rock, thanks for the consideration.

As for what percent... well, what percent from friends (hopefully not much :)), from work (again!), from random folks... well, who knows. From retailers and such, sure. But right now - what percent don't show it? Who knows, I'd have to cycle through every email... that's not going to happen. On the other hand... you might think, in the back of your mind... how to present an overall percent... I don't know how to even show this to the user, but to me it'd be fascinating to see that of the email that avoided my spam filters, 85% of today's mail had a tracker. Maybe even a "xx%" badge on the mail dock, or in one of the status lines... or...?

But to me it's not about the high-percent that have them, which is somewhat akin to asking what % is spam - 99%+? But Of the ones that manage to get through your defenses... I'd personally love to be able to mark or flag any and all that have this if I wish. It'd be especially interesting/pernicious if a sender that you thought you trusted (friend or biz or w/e) was flagged, that'd be a big deal. For the amazons/etc of the world, sure, that's noise.

I've been aware of this forever, but am interested in seeing how pervasive it is in places where I wouldn't normally expect it... for instance, the alert from github on your response - "MailTrackerBlocker blocked a tracker from GitHub." What?!? I may be naive, but I didn't expect it. And really, I didn't even see the alert until I went back while writing this. A fine line between informative and invisible, or something like that.

You say (probably correctly) that most mail does have them... but it does seem that having control of how to present that to the end consumer is critical to better understanding. Yes, you don't want to simply wash out everything in a big red smear.

But... wait a second... don't tell me.... Brutus, et tu?

[apparition47]

Definitely. Trackers are definitely way more pervasive than I think a lot of people realize and moreover, just as easy for for everyone and their neighbour to implement. @dhh (who inspired this project) has been in the spotlight a lot over this very issue. I've had coworkers send me emails with trackers too. It's unbelievable. And so MailTrackerBlocker operates on this very premise of trackers being so prevalent that as you said, after the 10th email, any sort of flashy or overbearing UI indicator would end up being noise.

But I've been considering minor changes to the indicator. Perhaps something along the lines of Ugly Email where they have a 👁️ icon on the Conversation List if a tracker is detected.

Your other points noted though for having options. Maybe I'll see if I can hook into the Rules with an extra condition for something like "Message has tracker". A summary/stats page like you said might be interesting too.

)

[raybrownco]

Yeah - great ideas, @zenfish!

@apparition47: if you would like me to generate any new assets related to this work, please let me know. I'm definitely interested in mocking up some summary/stat page concepts if and when you're ready to tackle that feature.

[ghost]

I'll second/third/fourth/whatever the request for an X-header. I have filters that run in the delivery pipeline that do this (I operate my own mail server). It makes writing rules (without mucking with Mail to add new rules conditions) really easy.

The content of the X-header can be JSON or whatever so the content can be quite complex. Most of the time you just care that the X-header is there or not but if you have something like MailSuite installed, they give you a bunch of rules conditions that allow you to deal very effectively with the X-header content and filter on that.

Since I'm a Unix guy, I'd also like to see your rules engine be packaged as a separate component that can be re-packaged in a delivery filter that runs on the mail server (Linux, FreeBSD, etc) way upstream from Mail. IMHO, what MailTrackerBlocker does belongs in the mail server anyway (sort of like what SpamAssassin or mailreaver do for spam). If I'm writing a sieve snippet to do this, it might look like this:

# standard
#
require ["fileinto", "imap4flags", "envelope", "variables"];

# dovecot pigeonhole-specific  
#    ('dovecot pigeonhole' is an implementation of RFC 5228 -- see https://tools.ietf.org/html/rfc5228)
#
require ["vnd.dovecot.filter"];    # include capability to run filters from sieve scripts

# pipe the message through the mail tracker blocker ('mtb') -- a program that encapsulates the 
#    MailTrackerBlocker rules engine (which adds X-MailTracker: header for later use)
#
filter "mtb" ["-f", "-"];               # THIS IS THE IMPORTANT STEP... 
                                        #      '-f -' tells 'mtb' to read the email msg from stdin
                                        #      'mtb' must always write to stdout

# Do something with the X-MailTracker header...
#
if allof(
            exists "X-MailTracker",
            header :contains "X-MailTracker" "some text"
        ) 
{
   setflag "\\Seen";       # marks the message as 'Seen'
   fileinto "Suspect";     # files the message into the 'Suspect' IMAP folder
   stop;
}

[ghost]

I'll second/third/fourth/whatever the request for an X-header. I have filters that run in the delivery pipeline that do this (I operate my own mail server). It makes writing rules (without mucking with Mail to add new rules conditions) really easy.

Please see leggett/simplify-trackers/ruby. I implemented a simple ruby filter that does exactly this.

[apparition47]

rules engine be packaged as a separate component

@caponecicero I like this idea. For your Ruby filter program, what does the email input/format look like? e.g. something standard like the eml files that come out of Mail with the headers and body?

[ghost]

You write:

@caponecicero I like this idea. For your Ruby filter program, what does the email input/format look like? e.g. something standard like the eml files that come out of Mail with the headers and body?

That's exactly right... The input/output format is a mail message. I put it into my delivery pipeline today to read the email from STDIN, modify it in-place if necessary, and write the (possibly-modified) email to STDOUT (that's what sieve requires of filters). If the message has trackers, they are all removed and a header called X-Trackers-Blocked: is added to the message with a list of the names of the trackers that were removed. The names are the keys in the hash defined in simplify-tracker-blocklist.js. If there are anonymous trackers discovered (an anonymous tracker a tracker that doesn't appear in the hash but is an <img> with height <= 1 and width <= 1).

The tracker blocker does it's job while the message is being delivered so it's not part of Apple Mail -- it's part of the email delivery pipeline (it happens before the message lands in the users's server INBOX). I have several of these kinds of filters for different things. They all work basically the same way and are all strung together with sieve.

The ruby code is pretty well commented so you can figure out what's going on without being a ruby expert (although that would definitely help). The mail and nokogiri gems do most of the heavy lifting.