Optimize fetch annotations

[elodieburtin]

Currently, when fetching annotations with GET /api/annotation.json, the request can be very long, even if the user specified the query parameter max limiting the number of results. In fact, this parameter does not seem to reduce the response time by much: when fetching from a collection having around 300,000 annotations, I obtain the following metrics on a local instance:

• with max=1, response time = 13,613 ms;
• with max=1000, response time = 13,810 ms;
• with max=100,000, response time = 14,122 ms. Would it be possible to optimize the request to reduce the response time, at least when max is small?

[urubens]

As far as I know, the issue is in fact common to all collection requests. max and offset parameters are never used in DB requests. All results are systematically fetched from the database and the results are then filtered with these parameters in the JSON response.

So, in these 3 requests, your 300,000 annotations are fetched from the database and then filtered, even for max=1 ! The variation is response time only corresponds to the difference in HTTP response payload size.