Redocly
$ref lines not resolving during bundle - is there a limit to the number of $ref lines you can use in the bundle function?
Describe the bug
I am using the bundle function to construct an Open API yaml file and am getting stuck with a single $ref line not resolving correctly & being updated to correct syntax in the bundled output file, no matter what I try. It seems that for whichever endpoint comes second in paths-base.yaml, one of the $ref lines just won't resolve. The bundled output contains the following:
If you swap the order of the endpoints being declared, this issue will swap to the other endpoint.
My only thought is that there is a limit to the number of $ref references the bundle function can handle?
To Reproduce Steps to reproduce the behavior:
- Import attached repo
- Run command
redocly bundle base.yaml --output bundled-api.yaml - See line 224:
$ref: ../../components/models/MatterListResponseNew.yaml
Expected behavior
The $ref should resolve like
$ref: '#/components/schemas/PublicDocsErrorResponse'
This only happens this this $ref, however if you swap the order of the endpoints in paths-base.yaml, this resolve issue will swap to the other endpoint.
Change:
/api/v1/cards:
$ref: ../paths/endpoints/GET-api-v1-cards.yaml
/api/v3/matters:
$ref: ../paths/endpoints/GET-api-v3-matters.yaml
to:
/api/v3/matters:
$ref: ../paths/endpoints/GET-api-v3-matters.yaml
/api/v1/cards:
$ref: ../paths/endpoints/GET-api-v1-cards.yaml
Logs
PS C:\repositories\openapi-spec-sanitized-fixed> redocly bundle base.yaml --output bundled-api.yaml
bundling base.yaml...
📦 Created a bundle for base.yaml at bundled-api.yaml 32ms.
This is on the latest version of redocly cli.
Please let me know if you require any additional details.
You should start with fixing https://github.com/nickcorby/redocly-bundle-ref-issue/blob/main/paths/endpoints/GET-api-v1-cards.yaml#L35
This is invalid OpenAPI because the $ref overrides the response object with the reference to your shared assets and ignores the 200 definition entirely.
I understand you're use of common schemas, but in this case, the OpenAPI Spec requirements are a bit verbose here. You need to define all of the response codes or use an extension x-. The Response Object is not a JSON Schema and doesn't support $ref alongside other keywords.
Hi @jeremyfiel thanks for taking the time to look at this and provide some feedback.
I know what I've written isn't valid OpenAPI on its own, but with the bundle function resolving all of the $ref lines, the output should be (unless I'm missing something). I'm not having any issues using the $ref within the responses object, as you can see in the code I've provided:
responses:
"200":
description: "200 response"
headers:
Access-Control-Allow-Origin:
schema:
type: "string"
content:
application/json:
schema:
$ref: ../../components/models/MatterListResponseNew.yaml
$ref: ../shared-assets/error-responses.yaml
becomes
responses:
'200':
description: 200 response
headers:
Access-Control-Allow-Origin:
schema:
type: string
content:
application/json:
schema:
$ref: ../../components/models/MatterListResponseNew.yaml
'400':
description: 400 response
headers:
Access-Control-Allow-Origin:
schema:
type: string
content:
application/json:
schema:
$ref: '#/components/schemas/PublicDocsErrorResponse'
For whatever reason, $ref: ../shared-assets/error-responses.yaml is resolving correctly, the $ref lines within error-responses.yaml are also resolving, but $ref: ../../components/models/MatterListResponseNew.yaml isn't. For the other endpoint, it resolves just fine:
responses:
'200':
description: 200 response
headers:
Access-Control-Allow-Origin:
schema:
type: string
content:
application/json:
schema:
$ref: '#/components/schemas/CardListJson'
'400':
description: 400 response
headers:
Access-Control-Allow-Origin:
schema:
type: string
content:
application/json:
schema:
$ref: '#/components/schemas/PublicDocsErrorResponse'
This doesn't seem to be related to any specific $ref line or model or similar - the syntax is identical for all endpoints & swapping the order of the endpoints will resolve the error in one endpoint and move it to another. In my mind, this points to an issue with redocly.
The solution may just be that I need to list all of the responses in each endpoint - I was going for DRY (we will be adding hundreds of endpoints) but it may not be worth it if it's causing these issues.
Thanks for reporting the issue, @nickcorby. We'll look into it when we have time. However, I'd still recommend you use the actual specification requirements on where $refs can be put. If you want our tools to help you to spot wrong ones, you can switch on the built-in spec-strict-refs rule. Moreover, it's generally a bad idea to use $refs alongside other fields (even though it's partially allowed) because it creates ambiguity regarding the inheritance.
Note: potentially related to https://github.com/Redocly/redocly-cli/issues/1907.
Hi @tatomyr I'm not sure if this is related to the initial issue -
I'm restructuring my base.yaml file to try and make everything more OpenAPI compliant & am consulting the openapi-starter repo for guidance.
I've noticed in the readme file for Paths (https://github.com/Redocly/openapi-starter/blob/main/openapi/paths/README.md) that breaking endpoints up by operation underneath the path is mentioned as a viable option:
However, when I adopt this structure in my base.yaml file, I get openAPI errors:
Code:
paths:
/api/v1/cards:
get:
$ref: endpoints/api-v1-cards/GET.yaml
post:
$ref: endpoints/api-v1-cards/POST.yaml
Errors:
Are these errors to be expected with this approach? Should I not not use this approach as recommended in the readme?
It's worth noting that the bundle function executes without issue, all $ref lines are resolved, etc. so there doesn't seem to be any actual issue.
I like this approach as it allows you to separate operations into individual files which I think is a lot cleaner than having them all in one, however having to live with these errors would be quite annoying.
@nickcorby here's the excerpt of the starter docs:
Only the "file-per-path" option is semantically correct with the OpenAPI Specification 3.0.2. However, Redocly's openapi-cli will build valid bundles for any of the other options too.
We do try to bundle APIs correctly regardless $refs are used following the spec or not. However, in your case it seems that something went wrong, and I'm not sure what yet.
BTW, if you are restructuring your code, you can consider using the split command which creates a structure that we generally recommend to maintain.
I'll try to take a closer look at your example a bit later.
@tatomyr thanks for clarifying - I guess the errors just come with the "filter-per-operation" option not being semantically correct.
Nice suggestion with the split command (thank you again). I tried it and it opted for the "filter-per-path" option, unfortunately! I was able to hack a "filter-per-operation" approach which is bundling correctly (and doesn't return any IDE errors):
base.yaml:
paths:
/api/v1/cards:
$ref: endpoints/api-v1-cards/base.yaml
endpoints/api-v1-cards/base.yaml:
get:
$ref: GET.yaml
post:
$ref: POST.yaml
endpoints/api-v1-cards/GET.yaml:
operationId: "Card_GetCards"
tags:
- Card
summary: "Gets a list of cards of a given firm.\r\nThe result only contains cards whose version is greater than lastRowVer."
parameters:
...
...
...
I'm not sure if this is technically "supported", but it does seem to work semantically, and it bundles without any issues that I can see.
Regarding the reported error, I haven't run into it since I cleaned up my $ref lines - they're all semantically correct now and I'm not referencing anything alongside other content within a property like this:
responses:
"200":
description: "200 response"
headers:
Access-Control-Allow-Origin:
schema:
type: "string"
content:
application/json:
schema:
$ref: ../../components/models/MatterListResponseNew.yaml
$ref: ../shared-assets/error-responses.yaml
I don't know if this is relevant or not - I can see a world in which my non-semantic references just broke something somewhere & it's technically a "bug" but may not be worth sinking that much time into.
@nickcorby Thanks for the active discussion. We will definitely review and improve our openapi-starter. Please feel free to ask any questions about product and functionality.
it opted for the "filter-per-path" option, unfortunately!
@nickcorby do you suggest that the per-path references still produce an error? I think the issue might stem from declaring the components section explicitly. Normally, you don't need to list your components, as they are referenced within the paths section and automatically resolved by the bundler itself.
@tatomyr no error from the per-path references or the split, I was just hoping for a per-operation split!
I'm not declaring my schemas property explicitly anymore which is working as you mentioned (being resolved & auto-added by the bundler), however I still need to declare the securitySchemas property within components, unless there's another way to do this?
i took another stab at this one and fixed up some issues. https://github.com/jeremyfiel/redocly-bundle-ref-issue
Now I only see one issue with the bundled.yaml file which is the same issue you posted about, the second endpoint doesn't resolve the response body correctly.
If i remove the $ref: /shared-assests/error-responses.yaml definition from the second endpoint, the bundle is successfully processed.
The interesting part is it seems to work correctly if you remove that definition from the second entry of paths-base.yaml. Even with the first entry and the error-responses.yaml definition still intact, the bundle is successful.
I would say this is a bug, but honestly, it's against the spec to use $ref here, so it's highly discouraged and it's been well documented in the newer updates to the spec from 3.0.3 onwards; fixing this issue shouldn't be a priority.
however I still need to declare the securitySchemas property within components
@nickcorby yes, you can declare securitySchemas in components, it shouldn't interfere with schemas.