Reconsider API spec autogeneration

[ztane]

This is a controversial thing and I am playing the devil's advocate but I've seen a great push towards FastAPI from Pyramid or sidelining Pyramid completely at our clients, connections and partners just because FastAPI seemingly makes it easier to evolve the API design rapidly with code. Everyone who is actually evolving the API is complaining about having to do the changes twice or thrice to the API document, view code and actual model handling. Only I seem to be defending the design, which might just be because I have never touched the YAML myself :D

Perhaps there could be some hybrid approach that would combine best of both worlds, making it possible to derive the API spec from code up to a point - not unlike how Sphinx Autodoc works for documentation

[zupo]

How other languages solve this is the other way around -> generate code from API spec.

Concrete example how we do it for the Elm frontend using https://app.kafkai.com/api/v1:

• have a Makefile command that generates the APi connector code, i.e. make openapi
• run this command in CI and fail if there are changes -> to catch "whoops I forgot to re-generate the code" errors
• the command is something along the lines of:

openapi-generator-cli generate \
      --input-spec openapi.yaml \
      --enable-post-process-file \
      --generator-name elm \
      --config ${./openapi-generator.yml} \
      --template-dir ${./openapi-generator-templates} \
      --output "$out"
elm-format --yes $out/src/

I have some rough ideas on how we could do this for Pyramid views, but the pain of manually changing views hasn't been high enough for me to start optimizing it away. But would definitely love to see concrete proposals, even if they are for generating YAML from Pyramid view code.

[zupo]

I am parsing/dumping YAML in another project with https://github.com/Fatal1ty/mashumaro, and I like it a ton. If we go down the generation path, this library seems like a good avenue of research.

[coler-j]

Yes, the duplication of effort without generation is painful.

[dz0ny]

Pydantic generates JSON Schema from models, which can be converted to open spec with a little of work.

[zupo]

The tooling now seems to have come along far enough that we can should be able to do this without too much wheel reinventing, just reusing the tools out there:

1. Use https://github.com/koxudaxi/datamodel-code-generator to generate models from openapi.yaml.
2. Use https://openapi-core.readthedocs.io/en/latest/extensions.html to unmarshall data into models generated by 1..
3. Profit!