Ignore folders in mail source

[krillin666]

Hey @seanbreckenridge how are you doing ? I’ve noticed that my promnesia indexing is taking longer than usual, and found out that it might be because it is indexing all my Trash folders from my different emails. Is it possible to exclude certain folder of indexing using the Thunderbird method ?

[purarue]

Ah yeah, for the imap portion I have this in my config:

# this is all custom, and may not work for you -- this
# is how I filter certain Junk/Trash paths from my mail

FILTER_PARTS: Set[str] = {".notmuch", "Trash", "Spam", "Junk"}


def filter_mail_path(p: Path) -> bool:
    pset = set(p.parts[1:])
    if any(f in pset for f in FILTER_PARTS):
        return False
    return True


class mail:
    # locally synced IMAP mailboxes using mbsync
    class imap:
        # path[s]/glob to the the mailboxes/IMAP files
        # you could also just do something like:
        # mailboxes = "~/Documents/mail/*@*"
        # to match any files in that directory with '@' in them
        #
        # to confirm this is matching your files, can do:
        # hpi query my.mail.imap.files -s
        mailboxes = MAILDIR

        # filter function which filters the input paths
        filter_path: Optional[Callable[[Path], bool]] = filter_mail_path

but since the mbox just loads the one huge file, you cant exclude extensions or paths as far as I understand.

Could exclude the trash in the export in the first place (though not sure exactly how to do that)

Will do an export and try to debug to see if the path is accessible in some way, so trash can be filtered

[purarue]

Hmm, trying to reproduce what you might run into, do you remember how you setup the export?

I selected my inbox like this in Thunderbird:

And then followed what I described here in the mail doc:

Tools > ImportExportToolsNg > Export all messages in the Folder > Plain Text Format

Edit: Or just right clicking and hitting Export remote folder, which includes all ~10000 messages in my account, not just the 2000 local cache.

That way is just my inbox and not trash/spam that are getting included. Are you doing something else, possibly because of the scheduling?

[krillin666]

Hmm, trying to reproduce what you might run into, do you remember how you setup the export?

I selected my inbox like this in Thunderbird:

And then followed what I described here in the mail doc:

Tools > ImportExportToolsNg > Export all messages in the Folder > Plain Text Format

Edit: Or just right clicking and hitting Export remote folder, which includes all ~10000 messages in my account, not just the 2000 local cache.

That way is just my inbox and not trash/spam that are getting included. Are you doing something else, possibly because of the scheduling?

Sorry for the late reply, I'm using the automatic scheduling feature from the addon. And thus I cannot see how to exclude certain folders. I'll try and see if your filter works !

[krillin666]

Hum, it is still indexing the deleted, trash, etc folders. Though I'm using mbox instead of IMAP

[purarue]

Filtering is only a thing for imap because i have the files laid out in my filesystem, its not all in a single giant mbox file

As far as I remember, the thunderbird addon exports the whole account into mbox, and last time I debugged I couldnt find anything like a 'folder' or 'path' in the metadata that described where the email was from. Will try doing an automatic export with thunderbird to see if I can find anything like that

[purarue]

Just leaving this here incase I want to try and reproduce this later, setup daily backups and going to leave thunderbird open so it'll do a backup sometimes in the next day (could trigger manually but want to make sure I'm not doing something custom there)

Edit: In accordance with the instructions I left here, I switched to just mail files for now, let me know if youre doing something different

[krillin666]

Yeah I was already using the just mail files though I don't see any option for the scheduling to exclude Trahs/Deleted folders

[purarue]

sorry this takes a while, I can only really do a test once per day since I don't want to do something custom and I want it to run with scheduling. I had it exporting to msf files before, I changed this option to export to mbox (I think? will test tomorrow):

The MSF files do include trash/junk etc., so maybe I could see about parsing those? It seems like a waste though, mbox is a standard while msf is very specific to thunderbird