asyncapi
explore how we can perform validation before sending messages
currently in javascript websocket client we have SendOperation component we dynamically extract all send type operations from AsyncAPI document and directly send message. But we can take this to another level by performing validation of message being send with help of libraries like ajv. So the client user do not have to learn how to validate messages before sending them.
Ref - https://github.com/asyncapi/generator/blob/master/packages/templates/clients/websocket/javascript/components/SendOperation.js
Approach - we can have a common messageValidator method in generated client which can maybe used in future for receive operations also with more modifications.
The only problem I am facing is if we go with this method the schema will be display in generated client and this will lead to unnecessary increase in generated code.
/**
* AJV validator instance for messages validation.
*/
static messageValidator(message, schemas){
const ajv = new Ajv({ strict: false });
const validators = schemas.map(schema => ajv.compile(schema));
const isValid = validators.some(validator => validator(message));
return isValid;
}
/**
* Sends a sendEchoMessage message over the WebSocket connection.
*
* @param {Object} message - The message payload to send. Should match the schema defined in the AsyncAPI document.
* @param {WebSocket} [socket] - The WebSocket connection to use. If not provided, the client's own connection will be used.
* @throws {TypeError} If message cannot be stringified to JSON
* @throws {Error} If WebSocket connection is not in OPEN state
*/
static sendEchoMessage(message, socket) {
const schema = [
{
"x-parser-schema-id": "<anonymous-schema-1>"
},
{
"type": "object",
"description": "Represents a notification to the user",
"properties": {
"type": {
"type": "string",
"enum": [
"info",
"warning",
"error"
],
"x-parser-schema-id": "<anonymous-schema-2>"
},
"message": {
"type": "string",
"x-parser-schema-id": "<anonymous-schema-3>"
}
},
"required": [
"type",
"message"
],
"x-parser-schema-id": "Notification"
},
{
"type": "object",
"description": "Represents the current status of the server",
"properties": {
"status": {
"type": "string",
"enum": [
"online",
"offline",
"maintenance"
],
"x-parser-schema-id": "<anonymous-schema-4>"
},
"uptime": {
"type": "integer",
"description": "Uptime in seconds",
"x-parser-schema-id": "<anonymous-schema-5>"
}
},
"x-parser-schema-id": "Status"
}
];
const isValid = PostmanEchoWebSocketClientClient.messageValidator(message, schema);
if (!isValid) {
throw new Error('Invalid message format for sendEchoMessage');
}
try {
socket.send(JSON.stringify(message));
} catch (error) {
console.error('Error sending sendEchoMessage message:', error);
}
}
/**
* Instance method version of sendEchoMessage that uses the client's own WebSocket connection.
* @param {Object} message - The message payload to send
* @throws {Error} If WebSocket connection is not established
*/
sendEchoMessage(message){
if(!this.websocket){
throw new Error('WebSocket connection not established. Call connect() first.');
}
PostmanEchoWebSocketClientClient.sendEchoMessage(message, this.websocket);
}
I am exploring more possibility to solve the schema problem. We also need to explore other libraries for validation of schema.
Approach 2 - We can create a Schemas file with generation of client and readme.md and then import in client the schemas and use it in validation. Each operation will have an schemas array with schemas for all the message it can handle. for e.g - sendEchoMessageSchema and many more schemas for all the operations.
We can use @hyperjump/json-schema for schema validation as according to me it is actively maintained and issues are addressed by maintainers.
Other option can be https://github.com/WaleedAshraf/asyncapi-validator it is already being used in nodejs-template
@derberg Approach 1 - I tried to do some Recursive programming for cleaning JSON Schema. Manual Recursion with Cycle Tracking
function cleanSchema(schema, seen = new Set()) {
if (!schema || typeof schema !== 'object') {
return schema;
}
if (seen.has(schema)) {
throw new Error('Circular Rref');
}
seen.add(schema);
const cleaned = {};
for (const key in schema) {
if (key === 'x-parser-schema-id') {
continue;
}
cleaned[key] = cleanSchema(schema[key], seen);
}
seen.delete(schema);
return cleaned;
}
Approach 2 - JSON-stringify + Replacer.
function cleanSchema(schema) {
try {
return JSON.parse(
JSON.stringify(schema, (key, value) =>
key === 'x-parser-schema-id' ? undefined : value
)
);
} catch (err) {
console.error("Schema cleaning failed due to circular reference.");
throw err;
}
}
I tested above both functions on slack document also and it is cleaning perfectly fine. But I am not really sure what if schemas is extremely nested or some kind of edge case that might be missed.
Now in the generated client we will have 3 files schemas.js, client.js and README.md. schemas.js will have all the operations with the schemas of the message. one example of schemas.js is below -
const sendEchoMessageSchemas = [
{}
];
const handleTimeStampMessageSchemas = [
{
"type": "string",
"description": "A timestamp with GMT timezone.",
"example": "11:13:24 GMT+0000 (Coordinated Universal Time)"
}
];
module.exports = {
sendEchoMessageSchemas,
handleTimeStampMessageSchemas
};
The main advantage of this approach is we have a single file with all the operation and respective schemas so we just need to import in client class and validation for not only send operation but receive operation will also become easy in future.
and what about traversing of schemas using what parser has, or adding such functionality there? https://github.com/asyncapi/parser-js/blob/master/packages/parser/src/custom-operations/resolve-circular-refs.ts
also, from technical point of view, we don't really need to perform "the cleaning" right?
and what about traversing of schemas using what parser has, or adding such functionality there? https://github.com/asyncapi/parser-js/blob/master/packages/parser/src/custom-operations/resolve-circular-refs.ts
No, the main problem we where discussing was regarding cleaning of schema only and traversing can be done easily using in-built JSON.stringify(schema, (key, value) =>key === 'x-parser-schema-id' ? undefined : value) but this will fail if their is circular ref.
@derberg I read about different schemas and I realized this issue is way more complicated than what I was thinking. First question in AsyncAPI doc users can actually write different types of schemas like JSON, AsyncAPI, Avro, Protocol Buffer etc. as mention here so how we will plan validation? Do we have to write all the separate message validators?
I have tried @hyperjump/json-schema but I'm unable to implement because I am getting error - Validation error: Error: Invalid IRI-reference: [object Object]. I tried to solve it but nothing is working out 😓. When I was using AJV it was simply working with flag strict:false but this library doesn't have any kind of this flag and this is validating everything with latest.
Now, I feel like AsyncAPI might look simple in start but when you dive deep things start to feel complicated 😅
@derberg finally I was able to implement validation using @hyperjump/json-schema library. But their is a catch in this specific library you have to also mention draft of JSON schema you are using so I implemented for draft 2020-12. For more ref -
static async messageValidator(message, schemas) {
try {
const { registerSchema, validate, unregisterSchema } = await import("@hyperjump/json-schema/draft-2020-12");
for (const schema of schemas) {
registerSchema(schema, "http://example.com/schemas", "https://json-schema.org/draft/2020-12/schema");
const result = await validate("http://example.com/schemas", message);
unregisterSchema("http://example.com/schemas");
if (result.valid) return true;
}
return false;
} catch (error) {
console.error("Validation error:", error);
return false;
}
}
also, from technical point of view, we don't really need to perform "the cleaning" right?
yes, we don't need cleaning from technical point of view. I just proposed it because of keeping end-user in mind.
default is below, and for starter follow below
'application/schema;version=draft-07'
'application/schema+json;version=draft-07'
'application/schema+yaml;version=draft-07'
@derberg how you have done magic in components I am also trying magic last 4 hrs but nothing is happening 😆. I am trying that babel hack which is converting ESM to commonJS but in case of @hyperjump/json-schema it is using pure ESM in source code. The error SyntaxError: Cannot use import statement outside a module. But I saw @asyncapi/generator-react-sdk is also implemented in ESM then how it is working?
what is here - https://github.com/asyncapi/generator/blob/master/package-lock.json#L373 does it comes when install @asyncapi/generator-react-sdk? Or some kind of diff dependency installed?
components uses imports/exports so basically ESM, and later produces compiled commonjs
you need to add babel dependencies and proper babel configuration
@babel/core @babel/cli @babel/preset-env
notice that package.json points to main executable "main": "lib/index.js", which is libfolder, notsrc`
look at: "build": "babel src --out-dir lib"
@derberg I did that exactly. But the problem is I think both @asyncapi/generator-react-sdk and @asyncapi/modelina internally provide support for both ESM and CommonJS. But @hyperjump/json-schema is purely written in ESM and no transpile in CommonJS also so no support. wdyt 🤔 ? By the way in my code compilation is also happening.