Support for cyclic references and object deduplication

[jsonbrobot]

trafficstars

This is a feature request to enhance JSON-B for 1.1 by supporting serialising objects with cycles using JSON-Pointers.

If you have a data structure

class Person {
  String name;
  Person marriedTo;
}

And the following 2 Person instances: John, marriedTo Lisa; and Lisa, marriedTo John;

Then serialising any of those via JSON-B will likely end up in a stack overflow.

The same problem appears if you have a data structure without cycles but the same Java instance to appear multiple times. When serialising this via JSON-B then the person 'John' might get converted to JSON multiple times. And when receiving this JSON on the other side we will end up with having 3 different instances of 'John'.

The solution we have in Apache Johnzon is to provide a mode to use JsonPointers to reference any subsequent occurences while serialising.

Person john = new Person("John");
Person lisa = new Person("Lisa");
john.marriedTo = lisa;
lisa.marriedTo = john;

We use a Johnzon specific JsonbConfig johnzon.deduplicateObjects. If you serialise with this config flag enabled then you'll end up with the following JSON:

{
 "name":"John",
 "marriedTo": {
   "name":"Lisa",
   "marriedTo":"/"
 }
}

Note the "marriedTo":"/". This value is actually a JsonPointer to the root object (John). And you can also create many deeper nested structures with it. Of course they are not that human readable friendly. But perfect for serialisation and deserialisation.

This is a feature we already implemented in Apache Johnzon. It works really well. https://issues.apache.org/jira/browse/JOHNZON-135 https://issues.apache.org/jira/browse/JOHNZON-141 https://issues.apache.org/jira/browse/JOHNZON-143

[jsonbrobot]

• Issue Imported From: https://github.com/javaee/jsonb-spec/issues/72
• Original Issue Raised By:@struberg
• Original Issue Assigned To: Unassigned

[struberg]

I got some good feedback recently. This works great in the Java world. But we should probably also evaluate how this can be transformed in JavaScript.

[aguibert]

@struberg can you elaborate on how you thin this should work? For example, how would JSON-B know that "marriedTo":"/" is a JSON Pointer and not a String value of /?

JSON-P 1.1 already supports JSON Pointers, but I was not able to build the example document you showed using JSON-P APIs.

Here is what I tried:

        JsonObjectBuilder bBuilder = Json.createObjectBuilder().add("name", "B");
        JsonObjectBuilder aBuilder = Json.createObjectBuilder().add("name", "A");
        bBuilder.add("marriedTo", aBuilder);
        aBuilder.add("marriedTo", bBuilder);
        System.out.println(aBuilder.build()); // {"marriedTo":{"name":"B","marriedTo":{"name":"A"}}}
        System.out.println(bBuilder.build()); // {}

It seems that JSON-P does not handle this correctly at the moment. I am surprised by the output I got from the above code.

If I look at RFC 6901, which is the basis for JSON Pointers, it makes no mention of using pointers within a document structure like you showed in the OP. Rather, JSON pointers are only able to point to locations in an existing set of concrete JSON data.

I tried to do some searching on this to see if it is possible outside of RFC 6901 and it doesn't appear to be. Here is the closest item I found: https://stackoverflow.com/questions/54467030/is-there-any-way-to-use-json-pointers-to-use-relative-path-in-side-json-string

[aguibert]

I think it's important to consider interop with other languages and libraries besides just JSON-B here. While Johnzon may be able to understand JSON like this:

{
 "name":"John",
 "marriedTo": {
   "name":"Lisa",
   "marriedTo":"/"
 }
}

If I try this in JavaScript, it isn't interpreted correctly:

obj = {
 "name":"John",
 "marriedTo": {
   "name":"Lisa",
   "marriedTo":"/"
 }
}

JSON.stringify(obj)
"{"name":"John","marriedTo":{"name":"Lisa","marriedTo":"/"}}"

JSON.stringify(obj.marriedTo)
"{"name":"Lisa","marriedTo":"/"}"

JSON.stringify(obj.marriedTo.marriedTo)
""/""

---

Likewise, JavaScript cannot serialize an object with a circular reference either:

john = {name: "John"}
lisa = {name: "Lisa"}
john.marriedTo = lisa
lisa.marriedTo = john

JSON.stringify(john)

VM1212:1 Uncaught TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'marriedTo' -> object with constructor 'Object'
    --- property 'marriedTo' closes the circle
    at JSON.stringify (<anonymous>)
    at <anonymous>:1:6

[rmannibucau]

Hello

There are three main options to handle it:

1. Johnzon one, you visit the object and if you encounter twice the same instance you replace it by the pointer and when deserializing you do the opposite. Indeed to work it ignores that logic for strings which is fine. Only issue is changing payload shape.
2. Json ref spec, but this one does not spec anything (only "should", no "must") and changes the payload keys which is worse than the shape.
3. Ignore already visited instances. This one is a no go IMHO because it does not gurantee the shape at all depending the visitor detail.

In terms of interoperability the 3 options are equivalent: they all need a lib everywhere. I used 1. at work with js without any issue relying on a jsonschema (https://www.npmjs.com/package/smart-circular does it with jsonpath for ex) and 2. has some impls too. But at the end nothing really portable or mainstream AFAIK.

Side note: jsonp does not handle it - i think it is ok cause it does not really have "references" - and your output comes from the fact a builder.build call resets its internal state (triggered by add), duplicate the builder instance to get whta you want ;).

[aguibert]

since there is no clear standard solution to this in the JavaScript community, I'm inclined to just cancel this issue until the JS community standardizes a solution around this (if ever)

In the meantime, it's fine for implementations like Johnzon to have their own way of doing things, but I don't think it has a place in JSON-B because there is no standards to interoperate with for other languages.

[rmannibucau]

There is an alternative: a callback to give the replacement at serialization and deserialization time.

Something along onCyclicReference[De]Serialization(context)