Make "old value" optional

[snej]

The delta format specifies that when a value is modified or deleted, the old value appears in its entirety in the delta. This can make the delta a lot larger, and it's not necessary for applying the delta, so I'm not sure why it's present.

The only purpose I can see is to detect a mismatch if the delta is being applied to JSON that doesn't match the original that it was created from. But this will only detect some mismatches, and it's a pretty expensive way to get only partial protection. In the use case I'm evaluating JDP for, there are already checksums that will ensure that the delta is applied to the correct source JSON, so the old values are unnecessary overhead and I'd like to omit them.

My initial thought is simply to emit 0 instead of the old value. This preserves conformance with the diff format, though of course the patch process won't work unless the implementation skips verifying the old values.

It happens that I'd need to reimplement JDP anyway (in C++ and possibly Go), so of course I can make whatever changes I want in my implementation; but it would be good to be in sync with your reference implementation, or at least get your buy-in on any changes to the data format. Thanks!

[wmathes]

@snej Looking for this as well. Have you found a good solution for this?

[ejdaly]

The old value is needed for "unpatch" and "reverse" operations.

The old value is not actually used to verify a patch (there's very little in terms of conflict resolution at the moment, see #170)

RFC6902 JSON Patch is a one-way system (only the new value is included in the patch, not the original value) closer to what you seem to be wanting (e.g. https://github.com/chbrown/rfc6902)

"checksums that will ensure that the delta is applied to the correct source JSON"

I think having the projection (just the relevant parts of the JSON that have changed) is more helpful, because it allows you write logic to decide if you want to apply (or perhaps, partially apply) the delta.

[snej]

We don't use "unpatch" or "reverse", but we do need the data size to be as small as possible since the purpose of using JSON diffs is to minimize network traffic.

I'm familiar with RFC6902, but the schema is extremely verbose, to the point where many real-world diffs would be significantly larger than the entire document, making it of little practical use.

We ended up modifying the data format for this reason.

[DexterHuang]

is there anyway to not generate the old value in the diff json now? feels like its just useless data (aka money) sent over the network in some uses case.. is there anyway to optionally turn it off?

[Martinnord]

Is there any possibility to only output the diff? The question didn't quite get answered.

Thanks.

[DexterHuang]

@Martinnord I end up just implementing a diff function myself, its faster then trying to find a existing solution haha

[Martinnord]

@DexterHuang Would you mind sharing that function? I would really appreciate it.

[DexterHuang]

sure but there might be bugs and its complexly unrelated to this package@@ my use-case is just generating as tiny as possible diff packages to syn client with server. if you need string or array index based diff feature its not here haha

export class JsonDiffHandler {

  static calculateDiff(old: any, newObj: any) {
    const diff = {};
    const deleted = [];
    for (const [key, val] of Object.entries(old ?? {})) {
      const newVal = newObj?.[key];
      // RECURSIVE OBJECT
      if (typeof newVal === 'object' && typeof val === 'object') {
        const recursive = this.calculateDiff(val, newVal);
        if (recursive) {
          diff[key] = recursive;
        }
      }
      // DELETE
      else if (newVal === undefined) {
        deleted.push(key);
      }
      // UPDATE
      else if (newVal !== val) {
        diff[key] = newVal;
      }
    }
    for (const key of Object.keys(newObj ?? {})) {
      // ADD
      if (old?.[key] === undefined) {
        diff[key] = newObj[key];
      }
    }
    if (deleted.length === 1) {
      diff['$d'] = deleted[0];
    } else if (deleted.length > 1) {
      diff['$d'] = deleted;
    }
    if (Object.keys(diff).length > 0) {
      return diff;
    }
    return null;
  }
  static apply(obj: any, diff: any) {
    if (!diff) {
      return;
    }
    obj = JSON.parse(JSON.stringify(obj ?? {}));
    for (const [key, val] of Object.entries(diff)) {
      if (key === '$d') {
        if (typeof val === 'string') {
          delete obj[val];
        } else {
          for (const key of val as any) {
            delete obj[key];
          }
        }
        if (obj instanceof Array) {
          obj = obj.filter(x => x !== undefined);
        }
      } else if (typeof val === 'object' && typeof obj[key] === 'object') {
        obj[key] = this.apply(obj[key], val);
      } else {
        obj[key] = val;
      }
    }
    return obj;
  }
}