Add normalized hash-based test IDs to draft2020-12/enum.json (POC for #698)
Context
This is a small proof-of-concept for #698, implementing option 3 from our discussion:
an MD5 hash of the normalized schema + test data + valid flag.
The goal is:
- Same test across dialects → same ID
- IDs change automatically when the semantics of a test change
- Fully checkable in CI by recomputing the ID and comparing
normalize.js
- Imported and adapted the AST-builder gist from @jdesrosiers.
- Added a handler for the dialect-specific keyword draft-2020-12/dynamicRef.
- Stripped top-level file: $id during normalization so registerSchema() doesn’t reject test schemas.
- Exposed normalize() to return a stable, dialect-aware, $ref-resolved schema representation.
load-remotes.js
- Recursively loaded all remotes/ JSON files and registered them under http://localhost:1234/....
- Added deduplication so the same remote URL is never registered twice.
- Ensured remotes only load for matching dialects.
add-test-ids.js
- For each test missing an id, normalized its schema.
- Generated an ID using md5(normalizedSchema + data + valid).
- Wrote the updated tests back to disk (in this PR: only enum.json).
check-test-ids.js
- Re-normalized schema and recomputed IDs for each test.
- Reported missing, mismatched, or duplicate IDs.
- Exited with error when inconsistencies were found (CI-ready behavior).
enum.json
- Added id fields to each test using the MD5-based formula.
- No other changes to schema structure or test contents.
How to run
# Add IDs to enum.json
node scripts/add-test-ids.js tests/draft2020-12/enum.json
# Verify IDs within draft2020-12
node scripts/check-test-ids.js tests/draft2020-12
For now, only enum.json has IDs. The checker will still report "Missing ID" for other files in draft2020-12
Questions for reviewers
If this direction looks good, I can:
Extend the tooling as needed based on feedback
Add a small GitHub Action that runs check-test-ids.js on the touched version
Then start updating more files, one small PR at a time.
This should have been opened as draft PR. I've converted it this time. Next time you open a PR that's a work in progress for the purpose of getting feedback, a draft PR is the way to go.
Hey @jdesrosiers, Really sorry for the radio silence! I'm currently drowning in university exams and intern interview prep, so it's been a bit chaotic over here. I'll need a couple days to review and fix up the mistakes. I skimmed through and saw your notes about defensive programming I'll polish it up and circle back soon. Thanks!
Hey @jdesrosiers, I’ve updated add-test-ids.js to incorporate the defensive programming points you mentioned. Some of the earlier checks came from initially making it work for a single file, so I’ve now refined and cleaned those parts up.
I’d really appreciate any further feedback when you get a chance.