Index PDF and MS Office documents

[mdecimus]

There is poor support in the Rust ecosystem for extracting text from PDF and MsOffice documents. Look for alternatives.

[dumblob]

On one hand this is a useful feature. I see though one important issue with it though. And it is so much important that there should be a checkbox to disable this feature.

Namely this "content indexing" is largely unreliable in practice. Documents are super wild in their formats, producers, etc. that one can not guarantee proper indexing of them. But it gets much worse realizing that only a tiny subset of document types/formats can be indexed. Once a user finds out a document got indexed she automatically assumes all (incl. every single existing format ever made on this planet) will be indexed and searchable. Nothing is more distant from reality than this.

To sum up:

1. content indexing is useful
2. content indexing has to be optional (default on - but see point (3))
3. content indexing has to be fully explicit in the user-facing UI - namely when searching a mail box there has to be a checkbox explicitly listing document formats to be searched through their content

[yodatak]

Hello we could use a interface of ripgrep or ripgrepall i use It for search in pdf and docs

[yodatak]

https://github.com/phiresky/ripgrep-all

[cptspff]

How about supporting attachment parsing with an optional external solution like TikaServer? It's easy to implement, battle-tested and relatively complete.

Also, it's probably a good idea to offer a "limit content indexing to known contacts" setting due to the inherently risky business of content scanning. It's my understanding jmap-server will handle contacts as well in the future - please correct me if I'm wrong.

[mdecimus]

@cptspff and @yodatak I want to avoid using external software (unless these are available as libraries) since that will require users to install and maintain yet another component. But I am going to take a look at Tika and ripgrep to see how they're doing it, perhaps they have some of their functionality available as a library. Thanks!