OpenAPI-Specification
OpenAPI-Specification copied to clipboard
OAI
Support for path parameters which can contain slashes
I see there are a few issues which ask for the opposite, e.g. this one https://github.com/swagger-api/swagger-js/pull/280 , but what I'm asking for is to add some option to the parameter, which would allow it to contain non-encoded slashes.
My use case: I have an API which allows any arbitrary path to be passed in, for example all of these:
-
/api/tags -
/api/tags/foo -
/api/tags/foo/bar/baz
Are valid paths. I tried to describe it as follows:
/tags{tag_path}:
get:
parameters:
- name: tag_path
in: path
required: true
type: string
default: "/"
However, https://generator.swagger.io encodes slashes in the path, so it doesn't work. So is there a way to describe my API? It feels to me like a perfectly valid use case, so I'm surprised it's not supported.
If it's indeed not supported, then one possible solution is to add some option for the path parameter, which would allow it to contain non-encoded slashes.
What do you think?
The request is not new, and came often around the request to support RFC 6570. At the moment, we've decided to not support it, though we'll probably re-evaluate in the future.
I see, thanks for clarifying. However, what were (are) reasons not to support it?
As with other features, one of the things we keep in mind is tooling support, and we felt that it would complicate the requirements for OAS 3.0 compliant tools.
@dimonomid Not allowing unescaped slashes in parameter values is for the same reason we don't allow optional path segments. Some tooling needs to be able to deterministically map a URL to an operation. With the current limitations it is easy to match the path segments in the URL to those defined in the operation because the path segments are fixed and the only ambiguity arises when a parameter value matches a constant, and IMO the constant should always win.
However, as soon as we introduce optional path segments then we have the following issue:
Given the URL path:
/foo/bar/baz
Any one of these paths could be a match (using RFC6570 syntax for illustration only)
"{/segments}"
"/foo{/segments}"
"/foo{/segments}/baz"
"{/segments}/baz"
"{/segments}/bar/baz"
"/foo/bar/baz{/faz}"
"/foo/bar/baz"
Which then requires us to introduce a priority based matching system, which can be really challenging to debug. Should we define a priority algorithm? Should the definition writer be explicit about priority? Should we use document order?
I think we do need to solve this eventually, as well as using query parameters to distinguish operations. But until we have a very well defined priority algorithm that is easy to understand, I believe we need to keep the constraints in place.
Which then requires us to introduce a priority based matching system, which can be really challenging to debug
Isn't this a problem that the REST frameworks already do solve? OpenAPI only has to follow their lead so the documentation matches the implementation, right?
My understanding is that some frameworks don't address it, some do, but many do it in different ways. There is no standardized algorithm for deciding on the right match. Even RFC 6570 punts on this:
Some URI Templates can be used in reverse for the purpose of variable matching: comparing the template to a fully formed URI in order to extract the variable parts from that URI and assign them to the named variables. Variable matching only works well if the template expressions are delimited by the beginning or end of the URI or by characters that cannot be part of the expansion, such as reserved characters surrounding a simple string expression. In general, regular expression languages are better suited for variable matching.
Sure, there's no standard. And it can be a problem. But in practice I rarely encountered this to be a problem. As a developer I think about valid URIs and try to stay away from any ambiguities: They are problematic in the implementation, not in the documentation, are they?
@t1 One of the goals we keep reminding ourselves of when updating this spec is we want it to be as straightforward as possible for people to build tools around it. If we define behaviour but leave ambiguities then we could end up with a ecosystem of tools that are not interoperable. That would not be good.
Yes, you an individual usage of OpenAPI can avoid these ambiguities, but ensuring everyone avoids them in the same way is not nearly as easy.
We do want to achieve this, we just couldn't fit it into the 3.0 timeline. I can assure you that when we designed the current URI template support, we made sure adding the / prefix would be an easy non breaking change. We will simply introduce a new style value.
I believe this misses an important point regarding REST APIs, however. It is critical, in fact, that we make our REST APIs discoverable. What does this mean? It means that our Paths are going to be variable and discovered at run-time. Using hypertext (HATEOAS principle) combined with a properly constructed OPTIONS response should make a REST API truly discoverable. Prescribing a specific path definition in the specification actually precludes this sort of dynamic discovery. Having a variable path enables this sort of discovery. It does mean that debugging may be more difficult. It also means that current tools may not be able to understand the specification, but both of those are implementation requirements not design requirements. OpenAPI, I presumed, was a design specification and that as such should be tailored to the needs of REST API design including fully Level 3 compliant (Richardson maturity model) REST APIs. Insisting on a fixed path like this does seem to go against these concepts. APIs we are developing are intended to be RMM level 3 compliant and discoverable. We also use OpenAPI and this creates a conflict for us. We'd hoped that in version 3 this would have been resolved.
@wohnemus OpenAPI has never had the goal of describing a dynamic hypermedia API. The idea of a static API description and hypermedia are contradictory goals. I spent 6 years building a hypermedia based system and had no need for something like OpenAPI.
Most people building what they describe as a REST API are not trying to build a hypermedia API. They are building a HTTP API, which OpenAPI does a good job of describing. OpenAPI v3 introduces the notion of linking, but it is a design time linking concept, not a runtime one.
There are a number of people in the OpenAPI community who would like to see more support for hypermedia style APIs. We may be able to do something to help that in a future release, but as I've said before, the last thing I want is to have OpenAPI do hypermedia badly. And considering OpenAPI is all about communicating to a client an exact description of how a HTTP API works, and hypermedia is all about following your nose, reconciling those two successfully is not going to be easy.
If you have ideas as to how client/server coupling can be kept at a minimum whilst still using OpenAPI, I'd love to hear it. If we can do it successfully, then I will be fully behind it.
@wohnemus if you want a fully dynamic HATEOAS-driven REST API, I encourage you to come over to the JSON Schema project and help us out with JSON Hyper-Schema, which explicitly has those goals as opposed to the static description goals of a service definition like OpenAPI. The two projects serve different purposes (which @darrelmiller nailed perfectly in his hypermedia API vs HTTP API distinction) and I think Hyper-Schema may be more suited to your needs. We are doing a lot of work on JSON Hyper-Schema right now.
I hope that eventually as JSON Hyper-Schema matures, OpenAPI will find ways to adopt it, but Hyper-Schema is definitely not there yet. We're still working out the best practices and features for hypermedia APIs, and it's probably best that we do that separate from any larger project targeting static description such as OpenAPI.
@darrelmiller I understand the goals of OpenAPI but it is also true that vendors like WSO2, IBM and many others are adopting OpenAPI as the standard for their tools. As an API developer I need to use such tools for REST APIs, including those that are dynamic. This creates that conflict. @handrews we'd love to join in the discussion there too. We are making use of the HAL-JSON specification for this purpose. The schema definition and design married with OpenAPI specification will help support dynamic APIs.
@wohnemus and anyone else interested in hypermedia description: we're in the final review phase for publishing Draft 06 for the next (probably) 2 weeks or so, but after that I hope to dig into Hyper-Schema a lot for Draft 07. Our main limitation with Hyper-Schema is that we have very few people who are really looking to use it in actual HATEOAS-driven APIs, so getting more actual potential users in the discussion would help us move it forward.
@darrelmiller: Maybe the spec should just ban ambiguities, so tools will mark them as errors. If somebody chooses to ignore that, some implementation may still work due to priority management... but that's completely outside of the standard (and utterly confusing).
Everyone, just so we're on the same page - the @OAI/tdc has decided this will not be supported in 3.0. This is not to say the requirements and needs are not clear, but we can't afford to reopen it for discussion. It's time for us to make the cut for what gets in and what not.
We encourage further discussions so we can see if/how this gets in future versions.
@handrews You'll be glad to know that Bill ( @wohnemus ) and I (and our teams), are heavily using JSON Schema and JSON Hyperschema in our API definitions and documentation - including in the dynamic description of what the API can do. We'll be coming over with great alacrity :)
@darrelmiller Exactly the approach I was about suggest. AWS API Gateway uses OpenAPI for its API descriptions and simply allows you specify an API to a certain depth and then have a path parameter described as
/seg1/seg2/{restOfPath+}
This basically says ... well it's obvious: "swallow the rest of the path". Yes, I know this isn't standard OpenAPI nor is it standard OpenAPI Extension syntax - but they saw the need and did it. It avoids ambiguity because, after that, you aren't allowed to create ambiguous paths.
IBM's API Connect also uses OpenAPI - but has not chosen to allow this feature.
@darrelmiller Darrel, first thanks for your efforts - a thankless task! I completely get your point that the level of dynamism we're going for doesn't require OpenAPI per sé, but, as the two product citations above imply, in order to get our APIs published and marketed (maybe that's a dirty concept here - sorry :)) we need to put them in a "standard" gateway and the developer portal facilities that those gateways may provide - and that usually means conforming to the best declarative standard out there - OpenAPI. So we end up being constrained by the number of extensions (and liberties) that the implementors add to OpenAPI.
Obviously AWS has taken liberties to get it to work, whereas IBM has constrained itself to the OpenAPI standard in this case, meaning we have to work around it (figure out your longest path, then describe a path parameters for each segment in that path - yuk).
So (and admittedly not having thought this through to the extent I know you'll be able to) would something "restrictive" like AWS's solution be possible as a first pass in, say, version 3.1?
@bassmanitram 6570 URI Templates support what you describe,
/seg1/seg2{/restofPath}
We can easily add this syntax by introducing a new style value. That's not a problem.
I do understand the need. I work on Azure API Management and we support templates like /seg1/seg2/* that does the same thing, and we also use OpenAPI to describe our APIs. So, we have the challenge of OpenAPI not being able to describe some APIs that our customers are defining.
Only using wildcards at the end of a path is one way to limit the ambiguities and it does cover a fairly significant number of use cases. However, it would be nice to find a way to address this in a more general way so that we can solve it for query parameter discrimination too.
Once we get 3.0 out of the way, you can be sure this issue will be high on my list of priorities.
@darrelmiller Re RFC 6570: Exactly. I mentioned AWS's approach simply as an example of a provider who facilitates the functionality in their product. Their syntax is clearly not 6570-compliant, more regex-inspired I suspect.
All understood (Azure API Management is in two weeks time for me - thanks for the heads up :)).
Thanks
@bassmanitram great! Please feel free to introduce yourself on our quiet but still functional mailing list or grab my email address out of the the current drafts on master and say hi :-)
So this is not going to be supported soon, and I have to resort to a workaround.
If I have a path /tags{tag_path} and I enter /foo/bar as tag_path, then the actual query request URL will be: /tags%2Ffoo%2Fbar. So, I just added support for that on my backend: the endpoint handler for /tags* urldecodes the path (which is %2Ffoo%2Fbar), and it becomes /foo/bar again.
Yes, a hack, but it works, and it's better than nothing. In my case, tag names can't contain the / character, so there's no conflict. Your mileage may vary, of course.
We took a different approach and changed our API to use query parameters instead.
@wohnemus I described a way of having hypermedia support with an extended variant of OpenAPI in #577. I just need to get to implementing it ...
@ePaul WE ( @wohnemus me and others from our org) could have written you first couple of posts in that issue - it's spot on for what we too are trying to achieve. Now need time to digest it and start commenting on it - first glance looks very promising.
I think opposition to this is resulting from a particular way of thinking about API descriptions.
Many OpenAPI descriptions are what I'll call derivative: they are generated from service implementations or other API description formats. These descriptions are not the primary source of information about an API but they are still used to drive tooling. Usually this is on the client side, in particular for producing documentation and client libraries. For these cases, supporting slashes in path template variable bindings adds no ambiguity, because users know what methods they want to call when they call functions in a client library. But explicitly acknowledging the possibility of slashes does allow API client libraries to correctly generate API paths when these slashes should be preserved and not escaped.
Many of us have a tendency to think of OpenAPI descriptions in a stronger, definitive sense. In these cases, OpenAPI descriptions are the primary representations of services and service code is generated from them. This is a great use of OpenAPI that I myself advocate, but I don't think it is appropriate to limit the expressiveness of derivative descriptions just so that definitive descriptions are implementable. (The relevant problem is that slashes in template variables can lead to ambiguous server-side path matching).
Instead, we should use linters to make sure that descriptions are implementable (when we need them to be) and be liberal in what we accept as valid OpenAPI.
Here is a specific proposal to address this. It supports current AWS API Gateway usage and a large class of Google APIs that are currently unsupportable by OpenAPI.
What's the status on this? I actually just came across this need for HTTP --> gRPC transcoding to work properly.
This is what i have from swagger : https://swagger.io/docs/specification/paths-and-operations/
My 2 cents for anyone sent here by Google - if you have control over the client, it's going to be a much easier short-term solution to drop the variable length parameter in a query param instead: /api/tags?path=foo/bar/baz
/tags:
get:
parameters:
- name: 'path'
in: query
schema:
type: string
Depending on your implementation, path might need some url decoding. A workaround for sure, but just a reminder to check if your use case allows you to tweak the client URLs a little, because that'll be a much faster implementation than waiting for OAS v4+.
Hey, is there any update related to it ? Or any possible deadline/version release with this feature ?
