kin-openapi
kin-openapi copied to clipboard
cache loaded remote schemas
when resolving components for loaded schemas and when external resolution is enabled, kin-openapi will read the remote schema, parse it and resolve it in order to satisfy the reference attribute in the calling schema, but it does so every time a reference needs resolving leading to unnecessary network traffic request for the same remote schema.
For example:
components:
schemas:
AnotherTestSchema:
type: object
properties:
ref1:
"$ref": http://localhost:7965/components.openapi.yml#/components/schemas/CustomTestSchema
ref2:
"$ref": http://localhost:7965/components.openapi.yml#/components/schemas/Name
In the above case, external resolution for components.openapi.yml is performed when parsing the fragment above only to be done again for ref2. This is recursive and if components.openapi.yml includes other references they are resolved the same way.
This very quickly leads to major network traffic which at the very least is unnecessary.
This pull request caches the loaded schemas in the swagger loader and returns resolved schemas from the cache once they are loaded.
Added tests to ensure the schemas are loaded only once.
I understand this can be a major PITA and am very curious how this is supposed to be handled by OpenAPI / JSON Schema specs? Surely caching of HTTP content should at least honor the HTTP cache headers? Are remote schemas expected to be assumed to never change?
Introducing remote refs immutability may break things, introduce races... so I'm leaning towards "these just have to be immutable". I just can't find the part of the documentation mentioning this?
@bonnyr Hi! I think this should get in. Do you mind rebasing? Thanks
Actually there now is a swaggerLoader.LoadSwaggerFromURIFunc
so just adding an implementation of LoadSwaggerFromURIFunc
that relies on an outside cache var (maybe behind a mutex) sounds like an appropriate solution and would make for a great example that others can derive from for their own cache semantics needs.
See https://pkg.go.dev/github.com/getkin/kin-openapi/openapi3#ReadFromURIFunc and specifically https://pkg.go.dev/github.com/getkin/kin-openapi/openapi3#URIMapCache which is used by default:
var DefaultReadFromURI = URIMapCache(ReadFromURIs(ReadFromHTTP(http.DefaultClient), ReadFromFile))