api-blueprint
api-blueprint copied to clipboard
apiaryio
Hypermedia
I am surprised that this is marked for 1B, imho there are other, more real-world use issues that would be better implemented in 1B and leave Hypermedia until 1C
I believe Hypermedia elegantly solves many real-world issues such as client decoupling, versioning (along with content negotiation), discoverability to name a few. It also directly contributes to better API design, maintainability and scalability.
The idea is to bake its support directly into API Blueprint so you do not even have to think whether you are hypermedia or not. For example this could automatically provide the location HTTP header where applicable or construct the link HTTP header etc. In cooperation with #10 this could go even further utilizing media types such as Collection+JSON, Siren or HAL.
Idealistically, should you choose to, I would like you to think about resources, relations and actions not HTTP verbs, URIs and such, but that is definitely even beyond the 1C..
@mikekelly, this is how I project it at this very moment:
In API Blueprint (Markdown), using some semantic assumptions:
- Define a resource with a name, id or an URI
- Describe the resource model media type (state, attributes)
- Describe relations between the model and other resource models
- Specify a transaction example with a hypermedia affordance provider media type content negotiation
The API Blueprint parser (or its harness) will take the care of injecting the actual model media type and relations (possibly even actions) into the hypermedia affordance provider media type.
Using part of @lukestokes foxycart API something along these lines (again, just sketching here):
# Resource User [User]
+ Model (application/json)
```
{
"first_name": "Qing",
"last_name": "woldu",
"email": "[email protected]",
"phone": "12345678",
"affiliate_id": 0,
"is_programmer": false,
"is_front_end_developer": false,
"is_designer": false,
"is_merchant": false,
"date_created": "2012-02-29T13:55:09-0800",
"date_modified": "2013-07-11T15:28:56-0700"
}
```
+ Relations
+ self = [User][] ... This User
+ stores = [Stores][] ... Stores for This User
## Retrieve a User [GET]
+ Request (Accept: application/json)
+ Response 200 (application/json)
[User.json][]
+ Request (Accept: application/hal+json)
+ Response 200 (application/hal+json)
[User.hal+json][]
# Resource Stores [Stores]
+ Model (application/json)
...
The API Blueprint parser would take care of injecting the user application/json model into the respective response media type in its resulting AST (and AST media types) e.g. do just a verbatim copy for [User.json][] or build the application/hal+json representation in the case of [User.hal+json][] using the model and relations.
Note I would also love to have Siren on board alongside with HAL in the first round. Considering other provides later. Also this concept can be extended to auto generate the Link HTTP header.
This is in a nutshell how I think about this issue at the moment. There are still few pieces of puzzle missing but that is alright for now. Please let me know what you do think and whether it can work at all.
Thank you!
Note for @kevinswiber : This is the topic we didn't have the chance to discuss at #apicraft.
I'll let smarter people than myself comment on the approach, but let me just say I'm really excited about seeing progress in this direction! Our sandbox supports both HAL and Siren, so I hope it can be a useful "real world" test bed for trying some of this stuff out. If I can help in a specific way, please let me know. Here are some of the keys I can think of as it relates to the needs client developer:
- Educate and enforce the importance of REST principles (think in terms of media types, response codes, caching concerns, resources, link relationships, etc and less in terms of remote procedure calls to URLs).
- Use visualizations in the documentation that help them navigate relationships to find the resources of the API they are interested in. As a real world example, we have users who only want to integrate with the API so they can generate and manipulate coupons. Building a documentation interface that helps them navigate to that relationship will be great.
One thing I was also considering is that not every API has to (or will be) a true REST API. If there's a clear way to distinguish between JSON RPC APIs and REST Hypermedia APIs, that would be fantastic.
Great work, Z. I'm really looking forward to where this goes.
@lukestokes I do agree in all points but the very first one – you are very qualified, if not the best, for commenting on this issue!
Regarding the clients – as we have discussed with @cainus its the clients where we need to improve the situation next. Not by telling them what they should do but pointing real life benefits as well as offer the best tools to consume our APIs. Now imaging sticking an hypermedia-enabled API Blueprint into a client SDK to configure it for your API and provide realtime request checks etc... (OK now I am totally going both off topic & off the roof :)
I think the only reasonable way of achieving broader adoption is to ship APIs where clients will code against them without even knowing what hypermedia is. This means we need media types that people can treat as plain JSON and they are still just as easy to work with and understand.
If I was you, I would pick the media types you initially adopt here carefully. If you offer something that is too complicated, and not intuitive enough - you are going to undermine your larger objective.
But I would say that, obviously. :trollface:
Mike, while I do agree with this line:
ship APIs where clients will code against them without even knowing what hypermedia is
However I do believe a broader adoption would be achieved through providing API SDKs and/or tighter framework client integration rather than finding the "holy grail" of media types. I treat a media type just as a mean to transfer the resource representation and, in some cases, hyperlink relations. Idealistically a client should be interested in (and exposed only to) actual resource data and the relations, not in the technical means of the communication itself.
In another words as a client you should only need to know your data are and where to get the next.
This is exactly what I would like to achieve with API Blueprint. In the first step to sweep the technical means of a hypermedia API under a nice and tidy Markdown carpet, so you do not have to really think HAL, Siren or anything like this – you just focus on your data and relations. Second do the same on the consumer's end – using an API Blueprint on client to hide the HTTP / transfer technicalities and provide directly the data and relations.
What do you think?
It's funny, prior to the RESTfest talk by @bsletten , I would have been worried about hiding things behind an SDK or client library. But now, if REST is truly used under the hood, I can see that the format and the datastore don't matter as much, as long as information can be treated as addressable resources which tell you what can be done next. His keynote video isn't up on vimeo yet, or I'd link to it.
@zdne
However I do believe a broader adoption would be achieved through providing API SDKs and/or tighter framework client integration rather than finding the "holy grail" of media types. I treat a media type just as a mean to transfer the resource representation and, in some cases, hyperlink relations. Idealistically a client should be interested in (and exposed only to) actual resource data and the relations, not in the technical means of the communication itself.
:+1:
We care more about the media types, because that's what we do. Most people just want to get the job done.
The difficult thing when supporting multiple hypermedia types on the same server is transforming capabilities between media types. If one media type only supports links while another supports controls, do you whittle down to the least common denominator? If so, why have additional media types? Do you provide enough information for capabilities of ALL media types supported? If so, do you just discard that information when rendering media types that don't support it?
These aren't very easy problems to solve in some cases. I'm curious what route @lukestokes took with FoxyCart.
Note for @kevinswiber : This is the topic we didn't have the chance to discuss at #apicraft.
Hey @zdne, sorry we didn't get a chance to discuss.
Here's how I've been looking at Siren APIs, in terms of documentation.
Things to define:
- Classes
- Link relations (for entities and links)
- Actions
Entities of a particular class may have certain properties, sub-entities, actions, and links. (Note: An entity can have multiple classes.) Defining this is important in documentation. I use class values to route responses to appropriate handlers on the client side. This allows the client to respond to the actual message from the server instead of setting an expectation at request time. For some, this is a different way of thinking, but it's an extremely powerful, reactive pattern.
Everything flows from the entity class documentation. Sub-entities have link relations that should be defined. Actions should be defined, and so should expected input fields. Hidden input fields, of course, should not be defined. The Siren spec instructs clients to resubmit hidden fields.
As far as functional documentation (docs that are able to submit requests), with Siren, this has to be dynamic. One of Siren's benefits is communicating available actions at request-time, based on resource state. Potential state transitions can change, and so, too, should the active docs.
These aren't very easy problems to solve in some cases. I'm curious what route @lukestokes took with FoxyCart.
We are just at the lowest common denominator right now. I wanted to include actions in our Siren support (and still plan to in the future), but we didn't want to go there until we finalized more important link relationships and property naming conventions.
I also want to support Collection, though I'd probably have to do an extension of it because we want to include properties of the collection such as total_items, returned_items, limit, and offset. In the future I may look into XHTML as well.
The formatting of the output, imo, isn't nearly as hard as properly designing the API in the first place. As long as the API has a good format layer in place, it's just a matter of moving data around. We did run into some funkiness with how things are done in JSON vs. XML, but for the most part, I think we came up with good approaches that wouldn't appear as compromises. Our hope in supporting multiple formats is that we can be a "real world" API for people to start testing their hypermedia tools against. More tools = more adoption.
Feel free to play around with it. I'd love any feedback you have: https://api.foxycart.com/docs
Any way, as it relates to this discussion, maybe it would make sense to include @mamund H-Factor ideas when thinking about the documentation system. Just a thought.
I'm just dipping my toes into this hypermedia ocean. The featured apiary apis look great, hadn't discovered it until recently! Apiary looks really powerful! @lukestokes the foxy cart api looks like it already supports hypermedia and consequently serendipitous api navigation discovery!
Anywho, as a background, I've been trying to socialize RESTful services and take it to the next level (of maturity) @ The Container Store. As a start, we've been using swagger to "document" our internal API's. Thats a great start, and it seems like a very similar approach to apiary. I've been thinking about how to make the simple API's hypermedia aware, and secondly how to make them discoverable once it is. Like @mikekelly said,
I think the only reasonable way of achieving broader adoption is to ship APIs where clients will code against them without even knowing what hypermedia is.
My 2 cents; considerations when it comes to hyper media, documentation and restful services
- The generation of the documentation cannot be deliberate, as in, distinct from the act of designing the API/service itself. So either the service can allow the generation of service information or vice versa. That way each side does not have the risk of being out of date. So the workflow is really important for adoption.
- Secondly, the whole notion of "documentation" falls flat with respect to a hypermedia API. By definition for a hypermedia API you should only need to document the entry points (as proposed by @zdne) the rest should be inferred as a result of navigation and discovery. Otherwise, IMHO it smells of the RPC style documentation that we got with SOAP, WSDL and its band of brothers.
Glad to help take this discussion forward.
@kevinswiber
We care more about the media types, because that's what we do. Most people just want to get the job done.
I'm in the latter camp for sure. Hal happened because I wrote a gist to show what I was building, then bunch of other people got interested so made a little website, then people started asking for a spec...
I don't care that much about media types - I care about helping people build effective, usable APIs.
There are some media types which I, as someone who's been around this space for a bit, feel are basically over-complicated and solving problems nobody's even sure they have - which is fine in and of itself, but when it comes to projects like this is worth highlighting as a risk for the reason you've outlined here:
The difficult thing when supporting multiple hypermedia types on the same server is transforming capabilities between media types. If one media type only supports links while another supports controls, do you whittle down to the least common denominator? If so, why have additional media types? Do you provide enough information for capabilities of ALL media types supported? If so, do you just discard that information when rendering media types that don't support it?
Don't get me wrong, I think pushing things further is worthwhile, but I think picking the low-hanging fruit first and then figuring out where to go next is a more reasonable approach. It sounds like this is exactly what @lukestokes has done with the foxycart API - I'll be interested to see where they go next.
@mikekelly
I care about helping people build effective, usable APIs.
Excellent point Mike! This is my aspiration with API Blueprint.
Again looking at FoxyCart HAL Browser: Basically my idea is to describe what is in the left pane in a Markdown syntax and let API Blueprint parser generate what is on the right. Without any prior knowledge of HAL or HTTP / REST.
Just tell me about your data and the relations. Simple.
@kevinswiber
Thanks for your feedback and tips on Siren, much appreciated!
Indeed if we decide to go for both the approach of @lukestokes with the smallest common denominator is the way to go.
Now, it seems to me the (choice of) (hyper)media type is really more important when you have one end open. Should there ever be something like an API Blueprint driven client consuming an API designed with API Blueprint then the whole (hyper)media type would be really just a 'technicality' under the hood.
Yes that sounds like a really good idea. fwiw, when I spiked the haltalk demo - this is what the link relation documentation ended up looking like:
http://haltalk.herokuapp.com/rels/signup
Yay! This looks almost like an API Blueprint:
signup
Create an account [POST]
- Request
-
Headers
Content-Type: application/json -
Body
{ "username": "fred", "password": "pwnme", "real_name": "Fred Wilson" }
-
- Response 201
-
Headers
Location: -----
-
Very helpful. Thanks for sharing!
@mikekelly
I'm in the latter camp for sure. Hal happened because I wrote a gist to show what I was building, then bunch of other people got interested so made a little website, then people started asking for a spec...
You're in both camps. Everyone has a stake in their work, and hopefully, the belief that it's helpful.
There are some media types which I, as someone who's been around this space for a bit, feel are basically over-complicated and solving problems nobody's even sure they have
Siren isn't an academic project. It came out of real work, as well. It was solving my problems. Now I'm helping others use it to solve their problems. Siren does have capabilities HAL doesn't. It's okay if you don't need them. Other people do. This conversation between us is a broken record that is being played on 3 different mailing lists. I'd like to show @zdne some respect and not pollute this thread with nerd battle. I'm happy to take this to the Siren list if you'd like to play this song together once more.
http://haltalk.herokuapp.com/rels/signup
@mikekelly @zdne
So a question I run into from time to time asks if something like this is actually over-specified. The usage of HTTP headers and status codes is already well-defined. You could really just say "Follow HTTP" for a lot of this.
Also, there are some things missing that I'd care about as a client developer. What about validation? When might there be an error condition? Are usernames unique?
I'm also starting to favor leading with description rather than HTTP, but that's mostly a formatting thing. I'd prefer to highlight "Creates an account" over "POST," because that's the bit client developers are really going to care about.
I'd like to show @zdne some respect and not pollute this thread with nerd battle
great, me too! It's not a personal affront, I'm just offering advice that I think makes sense for this project at this point in time.
Also, there are some things missing that I'd care about as a client developer. What about validation? When might there be an error condition? Are usernames unique?
Yep - there are many more things this description document could contain, haltalk is a really basic demo - it's not a finished product.
The main point of the demo is to show that the various transitions of an API can be broken down into, documented, afforded, and discovered via link relations. It's a no-nonsense strategy for making an API discoverable and moving the attention away from URL patterns.
Now, it seems to me the (choice of) (hyper)media type is really more important when you have one end open. Should there ever be something like an API Blueprint driven client consuming an API designed with API Blueprint then the whole (hyper)media type would be really just a 'technicality' under the hood.
I want to say yes, because this sounds interesting, and anything that sounds interesting is worth doing, in my opinion. Innovation is a big, messy pile of ingredients. Sometimes you just need to throw 'em in the cauldron and see what comes out.
Media types offer different capabilities, and that's where it gets tricky. No one cares too much about links vs _links. When you're looking at consuming an API, what you care about is features offered by the provider. I think we all agree on this.
The question then becomes, as an API provider and potentially someone who will create clients for various platforms, "Which media types help support the features I'm looking to implement?" This is where making a mistake really kind of stinks. You can start off with something minimal and try to bolt-on later, but eventually, the Frankenstein monster emerges... or you intentionally cripple the capabilities of your API. (Those words sound harsh, but it's not always so terrible.)
So the argument for starting with a media type that has the most capabilities is there. Hydra[1] is another media type I've taken a liking to lately. I need to examine it more closely, but I would submit Hydra as a candidate for another one of these "capabilities-rich" media types.
Of course... you could always come up with whatever format you think would work for you in this situation and birth something like application/blueprint+json. :)
You should also take a look at ALPS[2] if you haven't already. It's aiming to add supporting application semantics to APIs.
I feel like CCing some people since I'm talking about their work, and surely, they know more about their work than I do. /cc @lanthaler @mamund
Now... light that fire and get that cauldron boiling. ;)
[1] http://www.markus-lanthaler.com/hydra/spec/latest/core/ [2] http://alps.io
When I started thinking about docs, I figured people building their businesses on our platform will want more information rather than less. I've only got some twig templates in a twitter bootstrap theme, but I thought about having a short paragraph summary of the link relationship along with the affordances (we're a CRUD-heavy api so one relationship does multiple things via various HTTP methods, something OPTIONS can also help discover). Additionally, I want to show the property with detailed descriptions, types and constraints. We also want to show some example outputs in the various formats we support. It's a work in progress, but here are some examples:
http://api.foxycart.com/rels/stores http://api.foxycart.com/rels/transaction http://api.foxycart.com/rels/create_client
When it comes to embedded resources, I also include stuff from /rels/store inside of /rels/stores. Something to think about as users may work with a collection in a slightly different manner than a single resource.
I don't think there will ever be a "one-size-fits-all" solution to everyone's needs, but at the very least, focusing on link relationships instead of URLs will be a big step forward, IMO. As long as things are extensible so I could add in my own details, that would be great.
Really excited about where this is going, guys. Thanks for encouraging me with your efforts. :)
http://api.foxycart.com/rels/stores http://api.foxycart.com/rels/transaction http://api.foxycart.com/rels/create_client
Hey, I think this looks pretty good!
Capturing all the elements you have outlined there in a more abstract way would get something closer to the hypermedia version of WADL.
@lukestokes Thanks for actually implementing a real hypermedia API with public documentation. It's helpful for the whole community and hopefully helpful for FoxyCart, as well.
So the argument for starting with a media type that has the most capabilities is there. Hydra[1] is another media type I've taken a liking to lately. I need to examine it more closely, but I would submit Hydra as a candidate for another one of these "capabilities-rich" media types.
Thanks for the nice words and cc'ing me Kevin. Unfortunately, I think I don't have much to add at this moment. I just wanted to say that if there are any questions regarding Hydra I'm here to answer them :-)
I know this thread is getting quite long; good stuff here. I wanted to add one more thing to the mix when considering how to go about documenting a "Hypermedia API"
I've recently gotten into the habit of documenting the actions, not the objects. I have a (very) rough example posted in another repo[1]. Right now this approach takes a couple steps (not ideal):
- doc the media type
- doc the domain
- provide examples (combine the media type and the domain)
I think that is too unwieldy and can be greatly improved. However, my point here is to pitch documenting the hypermedia controls, not the resources. Its a very diff approach, but something I think can work well. It allows servers to decide their own URIs, their own representation details (what data and which controls appear in responses) and still gives client devs enough to write a working client (that recognizes both the data and the controls when they show up in a response).
Anyway, its just another approach to consider.
Cheers.
[1] https://github.com/apiacademy/class-scheduling/blob/master/docs/problem-domain.asciidoc#actions
Which media types help support the features I'm looking to implement?
More features in the media type does not translate to more features of a service. Compromising on media type "features" and instead, folding that complexity into something else (e.g. link rel documentation) does not make a service "inferior". It is a design decision.
more can be less.
Defending complexity by offering more "features" is a well proven sales technique, though - I'll grant you that.
@mikekelly I acknowledge your opinions. I have a feeling individual stubbornness on both our parts will prevent compromise. You admit HAL is intentionally minimalist yet claim anything more complex is a waste. For some reason, you think HTML is the only other media type that gets to play in the game. You haven't recognized any positive aspects regarding any other hypermedia type. That is not very constructive to the conversation. I get it. You want everyone using HAL exclusively. That won't happen, in my opinion, because of its minimalist design. I can program with GOTO all day long, but you know what? Sometimes I like IF-ELSE structures, too. You are a very loud minority, and your opinions, though acknowledged, are losing value.
@mamund This approach is very interesting. Are the actions for a link ID always static or could they change at runtime?