Pull parsing/resolving/dereferencing iterators

[brettz9]

Was wondering if you might be able to provide a pull parsing capability (as an ES6 iterator, asynchronously or synchronously as desired) so that one could parse, resolve, and dereference at the pace and extent desired?

[philsturgeon]

Not sure what you mean! Could you expand on this a bit, or should we close it down?

[brettz9]

It'd be nice to be able to create an ES6 iterator so one could choose whether and how to continue parsing/resolving/dereferencing (or perhaps bundling).

One of my use cases for incrementally doing so with a schema is to able to lazily load and render a responsive schema-driven UI. Incrementally parsed data can be rendered as available in the DOM in a web app, and an incrementally dereferenced/bundled schema can be used to guide search options as the remote schema data becomes available. Of course some information couldn't be used until multiple files were resolved, but let's say we had this for our schema:

{
  type: 'object',
  properties: {
    abc: {
      $ref: 'https://site1.example'
    },
    xyz: {
      $ref: 'https://site2.example' // Resolves to allow strings of format: 'date'
    }
  }
}

and for one data file:

{
  "abc": "hello",
  "xyz": "2020-05-16"
}

If the user clicked on a form for editing the JSON, opting to view "xyz", we could lazily load the schema (first showing it as a plain string perhaps, or giving a "loading..." message) to show the date within a date picker. There could be a web of interconnected schemas whereby one could view or edit these incrementally, but without having to load all of the indeterminate number of schemas that might interconnect.

With an iterator, one could easily control how or if to continue resolving/dereferencing such as to conditionally abort for whatever reason (e.g., if a site is not within a whitelist), or even to modify the schema.

It might look something like this (just a very rough concept), again, using the standard ES6 iterator protocol:

const it = $RefParser.resolve(mySchema);
let result = it.next();
while (!result.done) {
  console.log(result.value);
  /*
    {
      pointerSource: '#/properties/abc',
      reference: 'https://site2.example',
      resolvedSchemaChunk: {
        type: 'string',
        format: 'date'
      }
    }
  */
  // We could even potentially change the schema instead:
  // it.injectSchemaChunk({type: 'number'});

  // Continue (or conditionally stop)
  result = it.next();
}

And if following the ES6 iterable protocol, it could be used like this:

const it = $RefParser.resolve(mySchema);
for (const resultValue of it) {
  console.log(resultValue);
  /*
    {
      pointerSource: '#/properties/abc',
      reference: 'https://site2.example',
      resolvedSchemaChunk: {
        type: 'string',
        format: 'date'
      }
    }
  */

  // We could even potentially change the schema instead:
  // it.injectSchemaChunk({type: 'number'});

  // Continue (or conditionally stop)
  if (someCondition) {
    break;
  }
}

or, to get them all:

const resultValues = [...$RefParser.resolve(mySchema);]

This could be implemented with generators which yield the individual metadata result values one at a time (instead of immediately returning and/or applying all of them).

[philsturgeon]

Looking for pull requests for new functionality people are interested in. If you feel like adding it please reopen and send a PR.