add HATEOAS discoverability links from responses to enable graph traversal

[rossajmcd]

we can currently get a location header out of a POST response, we should enable other methods for discoverability

[rossajmcd]

We need to enhance the current offering in placeholder capabilities, including multi output threading in graph dependency resolution to refine transitions.

[colin-lamed]

This is related to #170

[colin-lamed]

I'll just add some thoughts on this.

One goal is to support this without choosing a particular standard, or confining an organizations interpretation of one. Some HATEAOS standards:

• HAL
• JSON API
• JSON-LD
• Collection+JSON
• SIREN

This probably rules out supporting some features, like HAL's curies syntax, at least for now.

Currently

Currently we have

:rsp {
  :headers {
    "Location" "/api/resources/${resourceId}"
  }
}

where the resourceId is extracted from the response Location header, and then enables calling any endpoint which requires a resourceId as input.

I'm not sure if HATEAOS endpoint walking would replace or enhance the current endpoint walking strategy. I have the feeling it is an alternative. We may also need to indicate the starting endpoint.

Desired

Annotate in the codex that a response returns links, which point to other endpoints documented within the codex. These links may be in header or in body.

The endpoint walker, then can follow the endpoint as returned in the response. It would not need to extract placeholders in the URLs, although there may be situations we this could be desired.

Self links may not be required to follow. It could be valuable to check that the link is not broken, but we would just be exercising the data-surface, not the endpoint-surface. We may also have cyclical links in the case of sims.

It would be nice to support the Link header, with format: ""<${nextUrl}>; rel=\"next\", <${prevUrl}>; rel=\"prev\", <${firstUrl}>; rel=\"first\", <${lastUrl}>; rel=\"last\"" Although this is probably a case of self links.

Syntax ideas

We could use JsonPath expression for JSON bodies (and XPath for XML) to identify how to extract the links from a response. E.g. "$._links.self" for HAL, "$.links[?(@.rel=self)].href" for Collection+JSON

e.g.

:rsp {
  :links {
    "$._links.self" {:get "api/resources"}
    "$._links.next" {:get "api/resources"}
    "$._embedded.resource._links.self" {:get "api/resources/${resourceId}"}
    "$._embedded.resource._links.update" {:put "api/resources/${resourceId}"}
  }
}

We should probably treat headers in the same way (will this mean we have multiple ways of defining headers? Note Link header is being deconstructed here - but using a made-up expression language...)

:rsp {
  :links [
    { :hdr-expr "Location" :get "api/resources/${resourceId"}
    { :hdr-expr "Link[rel=next]" :get "api/resources"}
    { :body-expr "$._links.next" :get "api/resources"}}
  ]
}

What about additional endpoint information. E.g. indicating optional? (an update link may depend on read-only state/permissions)

The format gets increasingly more complicated

:rsp {
  :links {
    {:hdr-expr "Location"} [:get "api/resources/${resourceId"]
    {:hdr-expr "Link[rel=next]"} [:get "api/resources"]
    {:body-expr "$._links.update"} [:put "api/resources/${resourceId}" :optional]
    {:body-expr "$._links.next"} [:get "api/resources"]
  }
}

We could simplify the expressions by mentally mapping the response to a json which includes the headers as well as the body.

 :links {
   "$.hdrs.Location" [:get "api/resources/${resourceId}"]
   "$.hdrs.Link[?(@.rel=next)]" [:get "api/resources"]
   "$.hdrs.Link[?(@.rel=prev)]" [:get "api/resources"]
   "$.body._links.update" [:put "api/resources/${resourceId}" :optional]
   "$.body._links.next" [:get "api/resources"]
}

Probably should include :doc for each link too.

Ideas? Feedback?

[rossajmcd]

Totally agree - should be media type agnostic. Other media types to consider at some point could be various flavours of HTML and EDN. Also agree that HATEOAS traversal is a strategy - ability to select a strategy would be desirable. Under 'Desired' I would like to explore the concept of the graph of graphs, inter API/service traversal. I think there is very likely value in looking into it. Broken link checking is very important in a graph world, especially where an API may be fanning out requests to other APIs. I think generally an exploration of OPTION starts to look like it might be becoming more important at this point. Finally I like the idea of federating out to well known mechanisms for extraction and not trying to create our own which maps to all of them... avoid the billion dollar Hibernate mistake.