Paged JSON support, 'highlight' and 'snippet' URL parameters

[Yetangitu]

This PR encompasses the following changes:

• changed JSON endpoint to allow paged json, use without page parameter (or page=0) to get unpaged result
• added snippets and highlight parameters to search url, use highlight=0 to disable search term highlighting, snippets=0 to disable snippet generation.
• added optional usjon module support

The first two changes can be used in combination with the Searx meta-search plugin to Recoll to enable Searx to search Recoll sites.

[Yetangitu]

The last commits add support for using more than one database by setting RECOLL_EXTRA_DBS to the path for one or more (colon-separated) xapiandb directories (analog to the way the Qt GUI handles multiple databases). The code assumes there to be a recoll.conf file one level below the indicated database directories from which it will pull the indexed top directories. When using the dir parameter to limit a query to a given directory it will only query those databases with matching topdirs.

The reason for implementing this scheme is to limit the total set size for directed queries as this increases performance (less data to search through) and flexibility (easier to add indexed directories).

[ghost]

Hi,

I merged the "Paged JSON support, 'highlight' and 'snippet' URL parameters" into https://opensourceprojects.eu/p/recollwebui/code which is where I (recoll dev) maintain the webui until koniu reappears.

I can't merge the extradbs thing because the assumption about the config directory relative to the db one is wrong. If dbdir is set, the xapiandb can live anywhere.

For the main interface, extrabds contains index directories to make it clear that the configuration parameters are ignored.

In your case, I would use a different environment variable, and list configuration directories, from which you can retrieve both topdirs and dbdir (if the latter is not set, the xapian index indeed lives in the xapiandb subdir of the config).