New version JSON API (version 1.1)

[JPBergsma]

I noticed that a new version of the JSON API is out, version 1.1.

Do we want to update the OPTIMADE specification to be compatible with this version ? I have not read it in detail, so I do not know if there are things we would need to change.

[ml-evs]

Thanks for pointing this out @JPBergsma, I've just read through the changelog and would be happy to present these highlights at the next meeting this week.

Some highlights of the 1.1 release that may be of interest:

• @-Members: from what I read, basically allows JSON-LD to be embedded with JSON:API responses, which may be useful to many of us (and for streamlining our semantic interoperability going forward).
• Error objects: errors links have been extended to allow implementations to point to a human-readable error page in the JSON:API error (see links->type).
• Profiles: seems to allow basically what we have done? i.e., sharing a particular usage of JSON:API across specifications although we adhere to JSON:API strictly, this would allow us to register an extension and e.g., add a new top-level field in our definitions. This might be super useful for Bob's gripe discussed in https://github.com/Materials-Consortia/OPTIMADE/issues/408, as we could add a top-level field optimade_ (would have to check naming rules) without breaking JSON:API compliance. We may even be able to get away with mandating that it is the first field in the response, which would enable the streaming-style approach suggested at the workshop. One other point is that query parameters are meant to be namespaced, so we actually break JSON:API with our response_fields, response_format and api_hint parameters (which I think was discussed at some point).
• describedby field for links, which adds the isdescribedby field to links. We could provide the OPTIMADE openapi.json here for links to other providers which could allow some clients to navigate more seamlessly through OPTIMADE APIs.
• Links objects follow RFC8288 seems to be that you can now link to fragments/anchors with links objects.

In terms of where we are currently non-compliant with v1.0 (see discussion at https://github.com/Materials-Consortia/OPTIMADE/issues/420), v1.1 could perhaps allow us to tackle this with profiles.

[JPBergsma]

One other point is that query parameters are meant to be namespaced, so we actually break JSON:API with our response_fields, response_format and api_hint parameters (which I think was discussed at some point).

Yes, this seems to be a rather big change compared to the previous version. So we may even have to wait until version 2.0 before we can adopt JSON API 1.1 .

[rartino]

@ml-evs I was looking over these and wondering if we need to define "optimade:"-prefixed versions of our query parameters and deprecate the old ones.

As best I can understand the JSON:API specification, there are four levels of "decreasing power" in what they are allowed to do:

• Specification
• Implementation
• Extensions
• Profiles

Profiles cannot define new query parameters. Extensions can, but only on the format ExtensionName:identifier. However, "implementations" are allowed anything as long as it contains at least one non-alpha-numeric character.

https://jsonapi.org/format/#query-parameters-custom

Do we want to make OPTIMADE an "extension" of JSON:API, or are we happy it is 'merely' an "implementation" (whatever that distinction means...) We could say that one is allowed to prefix all our parameters with "optimade:" in prepration of making OPTIMADE v2.0.0 an JSON:API extension. But, is there a reason to bother about this? I cannot say that I very much like the idea of extending the length of our query strings by a bunch of unnecessary "optimade:"...

[rartino]

@ml-evs

Also, could you clarify how Profiles could help the issue in #408 ? We have always been able to add top level fields with underscores. I thought the issue was that the outermost data structure in a JSON:API response is a dictionary that is "unsorted" and thus (in most implementations dealing with JSON data), even with an "optimade_" field, there is no way to ensure that it appears in the, say, first 10 kb of data.

(This IMO is a fairly major flaw with JSON:API that they won't be able to repair without a backwards-breaking change. The outermost data structure should have been a list, with the first element containing the meta dictionary.)

[ml-evs]

We have always been able to add top level fields with underscores.

I hadn't appreciated this; assumed this would be breaking for JSON:API 1.0 (I never checked this closely though).

I thought the issue was that the outermost data structure in a JSON:API response is a dictionary that is "unsorted" and thus (in most implementations dealing with JSON data), even with an "optimade_" field, there is no way to ensure that it appears in the, say, first 10 kb of data.

This is still the main issue, yes. I'm not sure this stops us mandating amongst ourselves that it will be the first key in the response; if up until now clients have not been able to rely on the order of keys from the server, enforcing one now doesn't seem like a huge deal, especially as it is just an annotation. Presumably this only adds the constraint that the transport of the JSON itself should not change the order?

[rartino]

We have always been able to add top level fields with underscores.

I hadn't appreciated this; assumed this would be breaking for JSON:API 1.0 (I never checked this closely though).

I have been thinking that what I said above was true based on early OPTIMADE discussions when we changed over to JSON:API, but looking more carefully at the standard text now, I think you are right and I am wrong about this: in v1.0 this "rule" is about implementation-specific query parameters, not members.

With that said, for v1.0 there is another top-level key that seems intended for this kind of info: jsonapi, which has a meta field where one can stuff arbitrary info. This seems the right place for a meta -> OPTIMADE: true type key that would be easy to identify the format from if present.

I'm not sure this stops us mandating amongst ourselves that it will be the first key in the response

It could be a suggestion, but is that worth anything if implementations cannot rely on it to identify the file format? Implementation-wise it seems easy but ugly to do, though (hack the string output from your json library to inject the key at the top...)

[rartino]

I'm looking at the JSON:API v1.1 question again as I am trying to synchronize our work on property definitions in #445 with the intent to be able to use @-members for json-ld in the future. The feature of generally allowing @ members anywhere in v1.1 would hence be useful.

I think all is ok if we just update the reference to JSON:API to v1.1 since it is meant to be forward compatible; perhaps with the clarification that OPTIMADE is a specification that adds requirements onto JSON:API on "implementation level", i.e., it is not a Profile, nor an Extension.

As was discussed above, on implementation level our query parameters seems fine per https://jsonapi.org/format/1.1/#query-parameters-custom

[ml-evs]

I think this was closed by #461 -- we didn't end up couching any of our additions in terms of the JSON:API v1.1 features but I think we're still somewhat compatible