Supporting $ref to relative path

[hhp21]

I have a Swagger 2.0 spec api/spec.json and inside there I want to have a reference to a local file api/defs.json for my definitions. So in spec.json I'm doing something like:

    "/message": {
      "post": {
        "parameters": [{
          "name": "message",
          "in": "body",
          "schema": {
            "$ref": "defs.json"
          }

where defs.json looks like

{
    "properties": {
      "text": {
        "type": "string"
      }
    },
    "required": ["text"]
}

But I just get #/paths/~1message/post/parameters/0/schema/$ref: Not a valid JSON Reference on init

I've tried several variations like:

"$ref" : "./defs.json"

This starts but fails with #: Reference could not be resolved: ./defs.json when a request is sent.

I have read that $ref supports internal and remote (with an absolute URL), but not been able to find any documentation on relative paths.

Here is a mention that this might be possible already: https://github.com/swagger-api/swagger-spec/issues/275#issuecomment-95232922 Here is a mention that it has just been implemented in swagger-js: https://github.com/swagger-api/swagger-js/issues/417

Are relative paths supported at all?

[whitlockjc]

JSON Reference resolution is handled by json-refs and there is a known issue for supporting this: whitlockjc/json-refs/issues/11. I will keep this open in here until this issue is resolved in json-refs.

[whitlockjc]

This should be fixed but there is currently a bug in json-refs/issues/24 that you need to be careful of. No circular remote/relative references? Things will work like you want.

[glen-84]

@whitlockjc,

It's not working for me. Am I doing something wrong?

http://pastie.org/private/0ozh5wm8li9qrrifl9cbua (swagger.yaml) http://pastie.org/private/6coezsiq2d9bmawl4yygg (definitions/models/videos/video.yaml)

[whitlockjc]

How are you loading swagger.yaml?

[glen-84]

With Swagger UI.

Also, it seems to request the main file 8 times when I reference the video file, but it doesn't request the video file itself.

[whitlockjc]

swagger-ui does not support relative references.

[whitlockjc]

I'm going to reopen the issue until I have unit tests showing it works but swagger-ui is not related to swagger-tools.

[glen-84]

Thanks @whitlockjc, I guess it's this one: https://github.com/swagger-api/swagger-ui/issues/1456.

[oliverhr]

It would be great to be able to run this example:

https://github.com/swagger-api/swagger-spec/tree/master/examples/v2.0/yaml/petstore-separate https://github.com/swagger-api/swagger-spec/blob/master/versions/2.0.md#referenceObject

[whitlockjc]

The problem we have right now with supporting this is that swagger-tools does not know where the original document was loaded from so doing relative reference resolution isn't possible without providing that context. swagger-tools uses path-loader and json-refs which both support relative references but they do it based on process.cwd on Node.js and window.location in the browser. Whenever the location of the loaded document is not known and does not conform to the aforementioned defaults, the related json-refs/path-loader APIs have an options.location option to fill in the gap.

That being said, to support this I would need to update all APIs in swagger-tools to support passing this option to json-refs/path-loader. It's not impossible but it would change the APIs a bit. I need to think about this a little more.

[coconitro]

I'm having a little tough time following, is this only an issue only with swagger-ui? I'm unable to get it to work using the swagger-node module loading swagger.yaml (completely in node, no browser in the chain at all)

Relevant snippet

swagger.yaml (snipped)

        default:
          description: Error
          schema:
            $ref: "error.yaml#/ErrorResponse"

error.yaml

ErrorResponse:
  required:
    - errors
  properties:
    errors:
      type: array
      items:
        $ref: "#Error"

Then when running swagger-node's validate I get this:

swagger validate api/swagger/swagger.yaml 

Project Errors
--------------
#/paths/~1events/get/responses/default/schema/$ref: Not a valid JSON Reference
Results: 1 errors, 0 warnings

[whitlockjc]

Your #Error reference is wrong. It should be #/definitions/Error.

[coconitro]

Good catch although it appears to be tangential. I changed errors.yaml as a test to be:

ErrorResponse:
  required:
    - errors
  properties:
    errors:
      type: string

[whitlockjc]

What's the error now? I doubt swagger-node supports relative references just yet since it uses swagger-tools which is known not to support relative references except under certain conditions.

[coconitro]

Sorry I should have been more clear. Its the same error as before.

#/paths/~1events/get/responses/default/schema/$ref: Not a valid JSON Reference

Under what conditions does swagger-tools support relative references? I could maybe force them or look thru swagger-node to figure out how to get them to work.

[whitlockjc]

The paths must be relative to process.cwd which is not ideal of course. So for swagger-node, your relative references would start with ./api/swagger. So if you had a users.yaml in api/swagger, you'd have something like this:

# ...
  $ref: './api/swagger/users.yaml#/definitions/User'
# ...

[glen-84]

Not sure if it's relevant, but what we're doing is using swagger-parser (3.0.0-alpha.5) to dereference our swagger.yaml file (with relative paths), and then we save it as swagger.json and serve that.

[whitlockjc]

swagger-tools dereferences references too but relative references weren't a thing until recently. When we started work on breaking up swagger-tools, we started swagger-core-api and it does work with relative references. In face, swagger-core-api does better validation than swagger-tools at this point. The reason I bring this up is swagger-tools will be deprecated in the future and will be replaced by swagger-core-api and per-server middlewares/plugins that build on top of swagger-core-api.

[diogeneshamilton]

@whitlockjc I use swagger-editor currently, and it seems that I must wait for it to be updated to use swagger-core-api in order to be usable with relative URLs. Am I getting that right?

[whitlockjc]

Yes. We are in the process of migrating swagger-editor to swagger-core-api. Also, supporting relative references for the swagger-editor will likely involve some other work since it's intended right now to be a single-file UI but this could change that.

[dkhanal]

I'm currently in the process of designing a fairly large set of APIs with various widgets/types reused.

I'd like the Swagger Spec itself to be decomposed into one spec file per path referenced through $ref in YAML. Then the referenced spec file itself reference JSON schema definitions all through HTTP-based relative $ref's -- since the hostname would be tier/deployment-specific:

So (in pseudo-code):

myApiDoc.yaml:
    /my/resource/:
        $ref: "swagger-sppecs/myResource.json"

------

Schema definition in swagger-sppecs/myResource.json:
{
    "properties": {
        "propertyA": {
            "$ref":"../schema/propertyA.json"
        }
    }
}

------

Schema definition in schema/propertyA.json:
{
    "properties": {
        "propertyB": {
            "$ref":"../schema/propertyB.json"
        }
    }
}

My questions are:

1. With Swagger-Tools, is it possible to decompose/refactor YAML into specs (it looks like it is possible with the new changes)
2. With Swagger-Tools, is it possible to refactor the referenced JSON schemas themselves to utilize relative ref's in composite widgets and types? Ideally, I would like the schema library to be hosted externally, since the schema definitions could be utilized across technologies and applications.

Any thoughts on the current/upcoming capabilities or recommended patterns around this?

[whitlockjc]

The underlying library swagger-tools uses for JSON Reference resolution (json-refs) supports relative references but to use it requires swagger-tools to know where the loaded Swagger document came from, which it doesn't, and it needs a way to pass this information to json-refs. To support this we need to alter all APIs to take an options object so that we can pass the location where the Swagger document was loaded from to json-refs. There is one situation where this can work now but it is not something work documenting: If your relative references are relative to process.cwd() in Node.js or window.location in the browser.

[dkhanal]

Thank you for the response. I was able to get the Node side to work by making sure the relative refs were off-of process.cwd() like you suggested. However, neither Swagger UI nor Swagger Editor is able to expand the ref'ed schemas. Swagger UI just doesn't work, while Swagger Editor only shows the referenced relative URL for the model.

Is there any way to get Swagger Editor (or UI) to expand the schemas into a more human-friendly view (like it would if the definitions were local and not ref'ed)?

Regarding your comment on window.location, does that imply that the docs would have to served essentially off of process.cwd() for both node and Swagger Editor/UI to work?

[whitlockjc]

swagger-ui is its own thing and they do their own stuff. swagger-editor uses swagger-tools but remember that references then become relative to window.location and I'm sure that the swagger-editor doesn't serve up any documents other than the one. swagger-tools supporting relative references is just part of this as the whole tooling ecosystem needs to support it properly.

I'll work on getting swagger-tools to have an API for specifying the document root so relative references work universally. Then it's up to swagger-editor and others to use the new support.

[dkhanal]

Thanks again for the response. So is your recommendation for now (until all tools in the ecosystem support the doc root) to just go with a single specification document with all references local to the doc?

Having one large YAML with thousands of lines of specification sounds rather monolithic to me -- so as a newcomer to swagger/tools, I would appreciate your insight on what the prevalent industry patterns are for composing a large set of APIs (essentially around reuse and physical refactoring of the specification document).

[whitlockjc]

What server environment are you using?

[dkhanal]

node + express

[whitlockjc]

So just Express and swagger-tools or something else? How does swagger-editor and swagger-ui get into the mix?

[dkhanal]

I dropped Swagger UI for its inability to reference externally ref'ed schemas.

I have a locally built Swagger Editor that I am hosting on a static server (different than where API would be hosted) -- just to use the UI portion of the editor. The editor portion is not used.

Swagger Editor worked great as long as the ref'd URIs were absolute. But I need them to be relative to support multiple deployment environments, and that's kind of where I'm hitting a roadblock.

The Node+Express side (even mocks) works just fine (even with relative refs).

Only alternative that seems to work with Swagger Editor/UI is to make the YAML spec a monolithic doc with no external or relative refs for schemas.

[glen-84]

@dkhanal Did you see my comment above?

Here is our Gulp file: https://gist.github.com/glen-84/f1bdda0bbe9c32da0d3f