data.world-py icon indicating copy to clipboard operation
data.world-py copied to clipboard

Re-implement API functions using requests

Open rflprr opened this issue 6 years ago • 1 comments

Retire swagger auto-gen in favor of simpler implementation using requests.

Start by getting rid of:

  • .swagger-codegen
  • _swagger
  • .swagger-codegen-ignore
  • swagger-codegen-config.json
  • swagger-swapi-def.json
  • Makefile (update_swagger_codegen target)

The pattern I would like to encourage is one where:

  1. Each module (*.py) file represents a section of the data.world api (e.g. projects, datasets, insights, etc). The ApiClient class, should aggregate all modules and serve as a point of access to all of them (e.g. ApiClient.projects, ApiClient.datasets, etc)
  2. Each function follows a naming convention (TBD) and maps positional args to endpoint path parameters and keyword args to body parameters (see: https://github.com/datadotworld/target-datadotworld/blob/master/target_datadotworld/api_client.py#L164) 2.1. If Content-Type needs to be defined (e.g. streams): TBD 2.2. If upload parts need to defined (e.g. multi-part file upload): TBD 2.3. If Accepts needs to be defined (e.g. sql): TBD
  3. A request session object defines common headers such as Authorization and User-Agent (see: https://github.com/datadotworld/target-datadotworld/blob/master/target_datadotworld/api_client.py#L37)
  4. HTTP 429 responses are automatically handled (see: https://github.com/datadotworld/target-datadotworld/blob/master/target_datadotworld/api_client.py#L366)
  5. All validation and business rules are delegated to the server, and requests exceptions can be thrown as is (avoid anti-patterns like this: https://github.com/datadotworld/target-datadotworld/blob/master/target_datadotworld/api_client.py#L192)

Additionally:

  1. Every function must include detailed docstrings (**kwargs documentation should point to the appropriate model doc, for example, https://apidocs.data.world/api/models/datasetcreaterequest)
  2. Every function must be unit tested. Tests must assert that: 2.1. HTTP request is composed with exactly the expected path, headers and body) 2.2. Function returns the correct values upon success 2.3. Function returns the correct errors upon failure

IMPORTANT: To maximize the benefits of these improvements, I would suggest breaking compatibility (i.e. release as v2.0).

rflprr avatar Apr 10 '18 20:04 rflprr