IQSS
OpenAPI definition endpoint
What this PR does / why we need it:
On Dataverse 6.0 Payara was updated and this caused the url /openapi/ to stop working and caused an exception on the server.
This PR restores the functionality by adding a new endpoint /api/info/openapi/{format} this endpoint will take json/yaml as format and return the specification. Thanks to @poikilotherm for the recommendation to move the source files to the /META-INF/ directory.
We will need to address the following issues on the future:
- The response of the API needs to be described with the @APIResponse annotation, otherwise they will not appear on the spec.
- There is an issue with a couple of endpoints that share the same path: /api/v1/admin/groups/ip/{groupIdtf} with /api/v1/admin/groups/ip/{groupName} and /admin/workflows/{identifier} with /admin/workflows/{id} these endpoints share the same path but one of them uses PUT another GET so we may have to find a solution and decide what kind of solution we do for these endpoints, also keep in mind when creating new endpoints that "paths must be unique".
- There are some other issues but I am still getting a grasp of what they are, some are just warnings but beside the 2 duplications are another 6 errors but it will all come down on how much time/effort wants to be put into improving the spec.
I think the most important takeaway of now would be that if you create an endpoint be sure the path is unique also if you can start adding the @APIResponse annotation to new endpoints it would be great 😄
- https://github.com/payara/Payara/issues/6369
We incorporated the SmallRye OpenAPI plugin on our POM (https://github.com/smallrye/smallrye-open-api/tree/main/tools/maven-plugin) which will generate files for YAML and JSON formats and deposit them on edu/harvard/iq/dataverse/openapi/ these files will be provided by this endpoint depending on the format requested.
Which issue(s) this PR closes:
- Closes #10236
- Closes #9981
Special notes for your reviewer:
We are still waiting to hear back from @JR-1991 since he is helping with the validation. The PR is ready for review but we are keeping it as Draft to prevent from anyone merging it until we hear back from Jan.
Suggestions on how to test this:
A new endpoint was added at /api/info/openapi/{format} where version can be json or yaml, the parameter is not case sensitive so it will take YaMl or JsOn. It will allow in the future to add more formats if they are added.
The maven plugin added will generate the files.
Is there a release notes update needed for this change?: Yes
coverage: 20.569% (-0.01%) from 20.58% when pulling ef267d039b180fcf409fc1e7235ddb3571e6af59 on openapi-definition-endpoint into 26be8e191fd31ca338c830eb2dd30daa12d431e3 on develop.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
Thanks @jp-tosca! Tested it locally using Docker and received the OpenAPI schema from the endpoint.
I have tested the OpenAPI schema using Speakeasy, and it did not complete successfully. These are the validation errors that I have received:
validation error: [line 104] validate-content-type - multipart schema must be an object
validation error: [line 1066] path-params - paths `/api/v1/admin/groups/ip/{groupIdtf}` and `/api/v1/admin/groups/ip/{groupName}` must not be equivalent, paths must be unique
validation error: [line 1940] path-params - paths `/api/v1/admin/workflows/{identifier}` and `/api/v1/admin/workflows/{id}` must not be equivalent, paths must be unique
validation error: [line 2639] validate-content-type - multipart schema must be an object
validation error: [line 2655] validate-content-type - multipart schema must be an object
validation error: [line 2671] validate-content-type - multipart schema must be an object
validation error: [line 3099] validate-content-type - multipart schema must be an object
validation error: [line 3183] validate-content-type - multipart schema must be an object
validation error: [line 4181] duplicate-operation-id - get_/api/v1/dataverses/{identifier}/metadatablocks/isRoot (GetAPIV1DataversesIdentifierMetadatablocksIsRoot) without 'operationId' in /api/v1/dataverses/{identifier}/metadatablocks/isRoot - get will collide with 'get_/api/v1/dataverses/{identifier}/metadatablocks/:isRoot' (GetAPIV1DataversesIdentifierMetadatablocksIsRoot) without 'operationId' [line 4155] when converted to method name
validation error: [line 4181] duplicate-operation-id - get_/api/v1/dataverses/{identifier}/metadatablocks/isRoot (getapiv1dataversesidentifiermetadatablocksisroot) without 'operationId' in /api/v1/dataverses/{identifier}/metadatablocks/isRoot - get will collide with 'get_/api/v1/dataverses/{identifier}/metadatablocks/:isRoot' (getapiv1dataversesidentifiermetadatablocksisroot) without 'operationId' [line 4155] when converted to file name
validation error: [line 4640] validate-content-type - multipart schema must be an object
validation error: [line 7303] validate-types - items attribute required for arrays in OpenAPI 3.0.X documents
validation error: [line 10770] duplicate-properties - hTTPUpload (HTTPUpload) will collide with httpUpload (HTTPUpload) [line 10714] when converted to field name
validation error: [line 10946] validate-types - items attribute required for arrays in OpenAPI 3.0.X documents
I'm happy to manually remove any troublesome endpoints and add multipart request bodies to test code generation. Also ditching schemes since none of those are used within the endpoint definitions. However, in the long run, I think it may be best to address these issues in the source code to ensure that we have a valid OpenAPI schema. Let me know how I can help!
@JR-1991 and I talked on Friday over Zullip about this and he provided the same test results for the original OpenAPI definition originally obtained from NC-DV.
Here is the log 🪵 : error.log
We can see that there is a reduction on the errors presented but we could set this a first-step solution where we re-enable the OpenAPI definition and we also provide now a YAML + JSON version vs the old one that defaulted only to YAML.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.
:package: Pushed preview images as
ghcr.io/gdcc/dataverse:openapi-definition-endpoint
ghcr.io/gdcc/configbaker:openapi-definition-endpoint
:ship: See on GHCR. Use by referencing with full name as printed above, mind the registry name.