TIDES icon indicating copy to clipboard operation
TIDES copied to clipboard

🚀💻 – Use Better Supported Approach for JSON Schema Validation

Open SorenSpicknall opened this issue 1 year ago • 0 comments

Describe the feature you want and how it meets your needs or solves a problem As a TIDES contributor, I want the project to adopt a JSON schema validation tool that's maintained and updated by a robust community, rather than the TIDES fork of an old GitHub Action that we're using right now.

Describe the solution you'd like A working solution should correctly validate data and validate schemas - at the time of writing, the frictionless-ci action only supports the former). It should also override strict interpretation of unknown keywords, a pain point for implementations based on the popular ajv tool that breaks usability for TIDES. At the time of PR creation, I am not certain that such a solution exists among popular GitHub Actions for JSON schema validation.

Describe alternatives you've considered At the moment, we are using a modified fork of this unmaintained Action. Our fork lives under the TIDES-transit GitHub organization, but does not have active stewardship - it continues to function for our use case, but should something break it is entirely on us to fix. This remains a workable, if finicky, option for the time being.

Additional context At the time the original json-schema-validator action broke for our use case and we created a forked version in the TIDES-transit GitHub organization, we concluded that it was prudent to create a slightly modified copy of the generic Frictionless table-schema.json in the TIDES repository. This copy references the draft-07 standard for JSON schema, rather than the draft-04 standard referenced in the example files from Frictionless. We chose to make this change because most JSON schema validation tools (including the popular ajv tool) don't support draft-04 any longer, and we wished to start futureproofing our project files. In addition to table-schema.json, our tides-datapackage-profile.json also references draft-07.

If Frictionless examples don't evolve to adopt draft-07 in the near term, this may have a downside of keeping TIDES slightly out of step with the generic Frictionless standards (hence needing to replicate the table schema file in the repository instead of fetching it directly from the standards site). The JSON schema features in use by TIDES at the time of writing aren't impacted directly by the move from draft-04 to draft-07, so the files referencing draft-07 could theoretically be reverted to draft-04 if a workable validation solution was found that required reversion to that earlier JSON schema standard.

SorenSpicknall avatar Oct 04 '23 20:10 SorenSpicknall