generator chore: add new package `@asyncapi/message-validation`

Description This is really basic setup for adding new package under generator/app which internally uses @hyperjump/json-schema. We are introducing it to abstract complexity of @hyperjump/json-schema and also it will be reusable across all clients message-validation so we don't need to write same code again and again.

Related issue(s) Fixes #1565

Jun 18 '25 14:06 Adi-204

⚠️ No Changeset found

Latest commit: 29c758c4c664d4ef002b38c6d2ea1c7e2f5e3106

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Jun 18 '25 14:06 changeset-bot[bot]

"""

Walkthrough

This change introduces the new @asyncapi/keeper package, a message payload validation library for validating messages against JSON Schema Draft-07, with supporting scripts, documentation, and tests. It provides utilities to compile schemas, validate messages, and integrate with AsyncAPI documents and operations, along with corresponding fixtures and setup files.

Changes

Cohort / File(s)	Change Summary
Monorepo scripts for keeper `package.json`	Added three npm scripts: `keeper:test`, `keeper:build`, and `keeper:lint` to run test, build, and lint commands filtered for the `@asyncapi/keeper` package.
Keeper package metadata and configuration `apps/keeper/package.json`	Introduced new `package.json` for `@asyncapi/keeper` defining metadata, dependencies (`@asyncapi/parser`, `@hyperjump/json-schema`), scripts (build, test, lint), Jest and Babel configurations.
Keeper package documentation `apps/keeper/README.md`	Added README documenting installation, usage, and API for the validation library, including usage examples and parameter descriptions for main functions.
Keeper Jest setup `apps/keeper/jest.setup.js`	Added Jest setup file to polyfill `structuredClone` using `@ungap/structured-clone` if not natively available.
Keeper .gitignore `apps/keeper/.gitignore`	Added `.gitignore` to exclude the `/lib` directory from version control in the keeper package.
Keeper validation module `apps/keeper/src/index.js`	Added main implementation for compiling schemas, compiling schemas by operation ID, validating messages, and removing compiled schemas. Handles AsyncAPI document parsing, schema registration, validation logic, and error handling.
Keeper utility functions `apps/keeper/src/utils.js`	Added utility functions: `parseAsyncAPIDocumentFromFile` (loads and parses AsyncAPI files) and `generateSchemaURI` (generates unique schema URIs).
Keeper test fixtures `apps/keeper/test/__fixtures__/asyncapi-message-validation.yml`	Added AsyncAPI YAML fixture defining a WebSocket API with a "hello" channel, message schema, and operation for use in integration tests.
Keeper integration tests `apps/keeper/test/index.test.js`	Added integration tests for core validation functions, covering valid and invalid messages, error cases, and operation-based validation using the AsyncAPI fixture.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Assessment against linked issues

Objective	Addressed	Explanation
Provide a way to validate messages against AsyncAPI-defined schemas before sending (Issue #1565)	✅
Integrate validation logic that can be used in client code to check payloads before sending (Issue #1565)	✅
Support validation based on operationId and message schemas from AsyncAPI documents (Issue #1565)	✅

Assessment against linked issues: Out-of-scope changes

No out-of-scope changes were found. All changes directly relate to the objectives of providing message validation capabilities for AsyncAPI-defined messages and operations. """

[!NOTE]

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.

✨ Finishing Touches

🧪 Generate unit tests

[ ] Create PR with unit tests
[ ] Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai generate unit tests to generate unit tests for this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

Jun 18 '25 14:06 coderabbitai[bot]

The current approach of using babel is most probably not going to work according to me because babel is compiling and converting the code for @hyperjump/json-schema/draft-07 but it's internally implementation is written in pure ESM and this library also doesn't provide support for CommonJS like @asyncapi/generator-react-sdk and @asyncapi/modelina.

Tests are also not written for now because Jest is not working with @hyperjump/json-schema/draft-07 as if you run npm run test it will throw error directly.

Jun 18 '25 14:06 Adi-204

@derberg I tried to do something similar to @asyncapi/modelina and now it is throwing below error.

I think main problem is -

Jest Assumes code in node_modules is CommonJS.
Skips transforming ESM code from dependencies like @hyperjump/json-schema

So we are transforming code inside index.js successfully but code present in node_modules of @hyperjump/json-schema is still in pure ESM hence it will throw error. wdyt 🤔? Does it make sense? I have also dropped message in JSON schema slack channel about some kind of workaround to solve this without going for "type": "module"

Current I am trying to add transformIgnorePatterns but it is also not working for now.

Jun 18 '25 16:06 Adi-204

I am currently trying to add

"transformIgnorePatterns": [
      "/node_modules/(?!@hyperjump/json-schema|json-pointer|pact)"
    ]

in package.json but it is also failing not sure why. People are having similar issues with other libraries also - https://github.com/jestjs/jest/issues/8272 not sure now whether it is problem with jest or hyperjump.

Jun 20 '25 06:06 Adi-204

simplify configurations a bit. I am currently using jest-esm-transformer instead of babel-jest. Now it is throwing error -

Now it is no more throwing error of unable to identify import statement. But now it is throwing error of unable to find @hyperjump/browser/jref module which is strange because I installed latest version of library, so all the other dependencies must be included. I am not using @hyperjump/browser/jref explicitly in my code and it is used in library here. I can also see @hyperjump/browser is peerDependencies when you installed @hyperjump/json-schema.

@derberg can you please look into it. This is strange error because dependency is already installed. Not sure why this error.

Jun 21 '25 06:06 Adi-204

app/message-validation/ node_modules

Screenshot 2025-06-24 095919

Global node_modules

Screenshot 2025-06-24 095949

I think this is classic problem of dependency version conflict in Turborepo. I tried to use moduleNameMapper to reference @hyperjump/browser/jref but it is also not working.

Jun 24 '25 05:06 Adi-204

@Adi-204 the problem is that @hyperjump/browser is not declared inside @hyperjump/json-schema as normal dependency but as a peerDependency which means you need to explicitly install it on your own. I guess the best version to install would be the one defined in peerDependency: https://github.com/hyperjump-io/json-schema/blob/main/package.json#L77

btw, didn't you think about some more fancy name for the library? payload-police or message-guard or events-shield 😄 message-validation is super boring 😆

Jun 24 '25 14:06 derberg

@derberg I tried this solution before also but when I am trying to install normally like npm i @hyperjump/browser but it is not adding explicitly to node_modules of message-validation and the error is still same. Do I need to install it differently 😕

Jun 24 '25 14:06 Adi-204

ok, the problem is jest related.

so explicit installation of browser package solves only one part of the issue

another issue is that it is installed in root node_modules, while json-schema package in message-validation. Looks like Jest do not understand export defined in package.json of browser lib, when package is not installed in the same node_modules as json-schema

"exports": {
    ".": "./lib/index.js",
    "./jref": "./lib/jref/index.js"
  }

just add "^@hyperjump/browser/jref$": "<rootDir>/../../node_modules/@hyperjump/browser/lib/jref/index.js" to moduleNameMapper in the apps/message-validation/package.json

Jun 25 '25 09:06 derberg

@derberg I also tried that approach as mention in comment it is also throwing internal error of the library.

and actually structuredClone is only mentioned in the file it is not even import but because it is internal implementation so it might be handle differently I searched in @hyperjump/json-schema codebase also https://github.com/search?q=repo%3Ahyperjump-io%2Fjson-schema%20structuredClone&type=code it is use only once. Should I again contact to Jason Desrosiers? wdyt 🤔 ?

Jun 25 '25 12:06 Adi-204

well the error is valid, structuredClone is rally not defined: https://github.com/hyperjump-io/json-schema/blob/main/lib/schema.js#L42, reach out to him, try slack or just create GH issue

Jun 25 '25 12:06 derberg

@derberg I looked into latest error throw found a discussion on stackoverflow - https://stackoverflow.com/questions/73607410/referenceerror-structuredclone-is-not-defined-using-jest-with-nodejs-typesc

Many solutions are written to solve the error we are getting.

I tried adding global.structuredClone mention on stackoverflow. This is working but it doesn't seem to be great idea to me.

I tried simple solution adding "testEnvironment": "node", because it is mention in official docs also that structuredClone should work with Node > 17 but I think [email protected] is not supporting this fully. so this really simple and nice approach failed. I tried to upgrade jest but it was throwing different error which I will try to solve later.

So, we do have a working solution but it is not best hence I am exploring more to find best and simple solution.

Jun 25 '25 19:06 Adi-204

@derberg as I mention in last comment, we had 2 solution -

Use https://www.npmjs.com/package/@ungap/structured-clone and we can remove the error of structuredClone as I have done in this PR. We are using @ungap/structured-clone only for testing and jest config, so I think it should not be a problem. But not sure, I think you insights are needed.
Upgrade Jest version to 29 and it was written at some places that in v.29 wont throw error of structuredClone. I tried this but only problem is if we upgrade to v29 it won't support jest-esm-transformer which is currently able to transform code. So we would have to go for babel-jest I tried again but it is unable to transform node_modules not sure why?

So, wdyt 🤔 ?

Jun 28 '25 14:06 Adi-204

@derberg if you didn;t like the package name we can change it maybe payload-police 😆

Jul 10 '25 09:07 Adi-204

how about lagertha as the name of the shield-maiden from Vikings tv series? You know, shield, event/message shield :) you know :D or valkyrie that were responsible for choose which warriors would die, and escort the worthy dead to Valhalla - you know, like validating against the schema who can pass the post :D

now depends which movies you are a fan off, Vikings or Avengers-Thor

Jul 11 '25 08:07 derberg

so @Adi-204 ? valkyrie or basically @asyncapi/valkyrie? 😃

Jul 11 '25 12:07 derberg

so @Adi-204 ? valkyrie or basically @asyncapi/valkyrie? 😃

@derberg actually to be honest I'm not big fan of any type of movies 😅

Jul 11 '25 12:07 Adi-204

any type of what I mentioned or in general? valkyrie is more related to nordic old school mythology, like greek or roman mythology that is very popular in mainstream. Good name that resonates well with this package scope. If not movies no than what is your thing? some specific gaming, or just code 🥰 😃

Jul 11 '25 13:07 derberg

any type of what I mentioned or in general? valkyrie is more related to nordic old school mythology, like greek or roman mythology that is very popular in mainstream. Good name that resonates well with this package scope. If not movies no than what is your thing? some specific gaming, or just code 🥰 😃

@derberg in general no movies. majority is code 🚀 but outside it any sports like chess, football and many more.

Jul 11 '25 13:07 Adi-204

then @asyncapi/goalie? goalie as short for goalkeeper, wdyt?

Jul 11 '25 21:07 derberg

ok, it is either @asyncapi/payload-police or @asyncapi/goalie or @asyncapi/keeper or @asyncapi/goalkeeper

Pick one and rename it accordingly

Jul 14 '25 10:07 derberg

@derberg package name updated.

Jul 14 '25 11:07 Adi-204

@derberg how you think of modified version. we can easily add many more validateBy in future if needed.

Jul 18 '25 13:07 Adi-204

@derberg I implemented error tech info after so many challenges and then when finally it was implemented I saw the error message logged not useful because you can see - https://github.com/hyperjump-io/json-schema/issues/56#issuecomment-2000539074 so we won't able to give tech error info to users for now at least 😞. I have not pushed that code because error message are not useful you can see the message share by users in the same issue I shared above. not sure what to do know I have not updated README.md yet just waiting for your confirmation. Error example -

"errors":[{
"keyword":"https://json-schema.org/evaluation/validate",
"absoluteKeywordLocation":"http://example.com/schemas/string#/items",
"instanceLocation":"#/0",
"valid":false
}]

Jul 24 '25 19:07 Adi-204

don't worry @Adi-204 I'm aware that in ecosystem there is no widely adopted standard for "human readable" errors. So the current error is completely fine. For initial implementation completely technical error log is enough.

there are projects like https://www.npmjs.com/package/@segment/ajv-human-errors or https://www.npmjs.com/package/better-ajv-errors but no need to use them anyway

Jul 29 '25 14:07 derberg