Core Client Requests and Comments

[mkhorton]

In no specific order:

• [x] Merge resters into a single class
• [x] ⭐️ Add task_id (List of task_id) to the search end point, to allow returning of fields for multiple specific task ids simultaneously
• [x] ⭐️ Ensure MSONable objects return the full object including @module, @class and @version, e.g. the structure field doesn't currently do this, and so monty_decode does not work
• [x] Make api_url simply endpoint for consistency
• [x] ⭐️ Ensure error msg is printed properly, e.g. try a query for a non-existent crystal system
• [x] ⭐️ Add "origins" and "task types" fields
• [x] ⭐️ Catch invalid criteria and raise an exception, don't just delete silently (e.g. currently, query the search end point with {'task_id:' ...} doesn't work, but it returns all documents, which is counter-intuitive)
• [x] To do: think about how to handle cases where mp-id changes and if this can be handled in a native and performant way
• [x] ⭐️ In the client Rester, make the search end-point a generator to automatically handle pagination, so that list(...) will just automatically make multiple queries until all results are returned (make the default limit to return all docs, but optionally can provide a limit=50)
• [x] Look at https://pypi.org/project/retrying/ and https://pypi.org/project/ratelimit/ for a more polite client
• [x] Look at https://locust.io for stress testing
• [x] ⭐️ Take a database version option to query collections for different database versions

Will add to these as I encounter them.

Starred the ones I think will be most useful for web development.

[mkhorton]

Later, we will need a stats API, which may require a build step too. This is for:

• [x] Stats for individual properties (e.g. histogram of densities in MP, so stats will give a list of values in histogram, with x_min and x_max to set scale), this will be necessary for the new search interface
• [ ] Overall database stats against time (adapting previous notebook to generate these plots), this is important for our home page and advertising what we offer ... this will be a lot more tricky, may be better to just hard-code for now, but a minimal API would still be useful (e.g. request materials, nanoporous, molecules etc. against time, get given back two lists, one for e.g. number of materials, the other for the date)

[mkhorton]

For the "merge resters" point for the end-user client, let's call this MPRester and see how compatible we can make it with current MPRester.

I'm not against breaking backwards compatibility (it'll probably be necessary to keep code clean), but we can at least try and make it as obvious as possible, e.g. maybe keep method names similar etc.