Alternative Preview Option with Markdown and MermaidJS Diagrams

[ivangsa]

Provide and alternative preview based on markdown that visualizes message payload as MermaidJS Class Diagrams

https://mermaid.js.org/syntax/classDiagram.html

• Support exporting this markdown preview as text so it can be included in external documentation
• It should support both asyncapi schema and and avro (.avsc files)
• It would be nice if scrolling between preview pannel and source code is kept in synch

[j-h-a]

I may have misunderstood the scope of this issue, it sounds like the idea is to have some kind of alternative preview mode. If mermaid diagrams could be inlined in markdown anywhere in the documentation would that cover this issue too? Or mean it's not necessary to have an alternative preview?

I've added an issue to asyncapi-react which would allow mermaid diagrams to be inlined in markdown.

Also, the scrolling sync thing sounds like an unrelated/separate feature :)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

[nikhilkalburgi]

Hi @ivangsa,

I am Nikhil, SWE at Daimler Trucks. I find this task very interesting and would like to contribute here as part of GSOC2024. Before I share my proposal with you, I have few questions to clear:

1. Do you want me to create a separate Preview Panel apart from the existing one with asyncapi-react preview?
2. The Message Payload contains mainly the type and properties where properties can be represented by class diagrams. But, want to understand whether I should show the relationship between message payload and other entites in the asyncapi document? Example:

channels:
  userSignedUp:
    description: Channel for user sign-up events
    subscribe:
      message:
        $ref: '#/components/messages/UserSignedUp'

components:
  messages:
    UserSignedUp:
      description: An event describing that a user just signed up.
      payload:
        $ref: '#/components/schemas/User'

    User:
      type: object
      properties:
        id:
          type: string
        fullName:
          type: string
        email:
          type: string
          format: email
        age:
          type: integer
          minimum: 18

The above Message Payload refers to the Users component and the below is the class diagram view:

classDiagram
  class User {
    id: string
    fullName: string
    email: string
    age: integer
  }

Should I only pick all the possible Message Payloads from the Asyncapi Document and show it like above or also the relations need to be shown.

If you can share more info on this then it will help me to follow up. Thanks in Advance!

[j-h-a]

Not sure if you saw already, but I previously made a PR that adds mermaid support. It worked just fine, you would just include the mermaid in any description markdown and it rendered inline in the preview.

However, there were some suggestions for improvements (a different approach that I didn't understand, embedding the mermaid code instead of referencing out to mermaid.js - I'm not a JS/React developer so I didn't really know how to implement it), and they were in the middle of trying to land a long-lived release branch and trying to get everything sorted, so instead of fixing and merging that PR they just said to make a new one on the new version-branch.

In any case, it really didn't seem very important to them, despite being super powerful as a documentation tool, so you might not get much support... but, if you do want to get this working that PR is probably a good place to start. It shows a working solution (albeit on an old version/branch) plus the feedback comments of what structural changes were needed.

[ivangsa]

Hi @nikhilkalburgi Yes, I mean a separate pannel that contains just makdown as source code, and renders as web pannel using vscode markdown support

Regarding mermaid representation, create the representation you would like to see as an user.. I would definetely do the payloads schemas with relationships... and more

@j-h-a I'm sorry you got a PR waiting so long.. I think these diagrams are very expressive... Any way, that PR is on a different component, that uses React... this is about plain markdown+mermaid

[nikhilkalburgi]

Hi @ivangsa,

Thank you for the quick response. I will engage into this and share you my proposal for review asap.🙂

Hi @j-h-a, thanks for your inputs

[josephinoo]

Hi @ivangsa, I'm interested in being part of this project. Could you provide me with more relevant information to get to know it better? I did something small.

flowchart TD
    subgraph "Streetlights Kafka API"
        subgraph "Servers"
            scram-connections["scram-connections"]
            mtls-connections["mtls-connections"]
        end
        subgraph "Channels"
            lightingMeasured["lightingMeasured"]
            lightTurnOn["lightTurnOn"]
            lightTurnOff["lightTurnOff"]
            lightsDim["lightsDim"]
        end
        subgraph "Operations"
            receiveLightMeasurement["receiveLightMeasurement"]
            turnOn["turnOn"]
            turnOff["turnOff"]
            dimLight["dimLight"]
        end
        subgraph "Messages"
            lightMeasured["lightMeasured"]
            turnOn["turnOn"]
            turnOff["turnOff"]
            dimLight["dimLight"]
        end
    end

    scram-connections --> lightingMeasured
    scram-connections --> lightTurnOn
    scram-connections --> lightTurnOff
    scram-connections --> lightsDim

    mtls-connections --> lightingMeasured
    mtls-connections --> lightTurnOn
    mtls-connections --> lightTurnOff
    mtls-connections --> lightsDim

    lightingMeasured --> receiveLightMeasurement
    lightTurnOn --> turnOn
    lightTurnOff --> turnOff
    lightsDim --> dimLight

[ivangsa]

Hi @josephinoo I love this flow chart.. At first I was thinking about representing only the schemas and their relationships but this looks awesome..

Can you generate this programatically? parsing an asyncapi.yml file and returning this output as a string...

[nikhilkalburgi]

Hi @ivangsa, I hope you are doing well. For fast few days, I am working on this project idea and have witnessed the following results: 1. Converting the asyncapi.yaml to Markdown and Rendering it on webview panel:

1. We need to install markdown-it npm package to render the markdown code to HTML
2. Parse the asyncapi.yaml to JS object with @asyncapi/parser
3. Like markdown-template, we need to implement logic to translate individual objects in asyncapi to makdown

Example: Info Object translated to Markdown Here, I built info( ) function to convert to md

Sample Code:

export function info(asyncapi:any) {

    console.log(asyncapi);
    const info = asyncapi.info();
  
    const defaultContentType = asyncapi.defaultContentType();
    const specId = info.id();
    const termsOfService = info.termsOfService();
    const license = info.license();
    const contact = info.contact();
    const externalDocs = info.externalDocs();
    const extensions = info.extensions();

    const infoList = [];
    if (specId) {
      infoList.push(`Specification ID: \`${specId}\``);
    }
    if (license) {
      infoList.push(license.url() ? (
          `License: [${license.name()}](${license.url()})`
      ) : `License: ${license.name()}`);
    }
    if (termsOfService) {
      infoList.push(
        `[${termsOfService}](${termsOfService})`
      );
    }
    if (defaultContentType) {
      infoList.push(
          `Default content type: [${defaultContentType}](https://www.iana.org/assignments/media-types/${defaultContentType})`
      );
    }
    if (contact) {
      if (contact.url()) {
        infoList.push(
            `Support: [${contact.url()}](${contact.name() || 'Link'})`
        );
      }
      if (contact.email()) {
        infoList.push(
            `Email support: [${`mailto:${contact.email()}`}](${contact.email()})`
        );
      }
    }

    return (
        `
# ${info.title()} ${info.version()} documentation
    
${
  infoList.map((value)=>{
    return '\n* '+ value;
  })
}

![${info.title()}](${(extensions.get('x-logo'))?extensions.get('x-logo').value():null})
    
        
#### ${info.description()}
      `);
}  

2. For mermaid Support

1. Downloading the mermaid.js npm package
2. moving the package to dist/node_modules
3. Adding the mermaid.min.js to localResources of the webview api and injecting it as <script src="<mermaid_path>"/> to panel.html

<script src="he/Path/In/Your/Package/mermaid.min.js"></script>
<script>
      mermaid.initialize({ startOnLoad: true });
</script>

3. Need to implement the Logic to convert the other objects like messages, servers, channels, operations to mermaidjs diagrams like flowchart shared by @josephinoo. Creating a function that uses all these objects and build the markdown for the diagrams. Example: I have not built the function yet, but here I have shown how the webview can render flowchart from @josephinoo in webview panel.

4. Apart from this, we can add copy button to export the generated markdown and also use avro-parser from asyncapi itself to convert the avro syntax as well and I had raised a PR to keep the preview in sync with asyncapi #206 (Merged) that can also work for this new preview

Eventually, I am trying to render the whole asyncapi.yaml to markdown + create mermaid diagrams by consolidating the info from all the required objects.

I have also shared a proposal with you on your slack Dms and would like to have your feedback to continue with you and contribute to the project.

[josephinoo]

Hi @nikhilkalburgi,

I've created a prototype that obviously needs improvements, but I'm working on it. @ivangsa, I'm figuring out how to enhance it and I want to understand a few more things to make it perfect. Also, I'm considering changing the color in the different subgraphs to make the information more visible. I will be working hard ..... Code:

import * as fs from 'fs';
import * as yaml from 'js-yaml';

interface AsyncAPIDocument {
    asyncapi: string;
    info: {
        title: string;
        version: string;
        description: string;
    };
    servers: {
        [key: string]: {
            host: string;
            protocol: string;
            description: string;
        };
    };
    channels: {
        [key: string]: {
            address: string;
            description: string;
            messages: {
                [key: string]: {
                    $ref: string;
                };
            };
            parameters?: {
                [key: string]: {
                    $ref: string;
                };
            };
        };
    };
    operations: {
        [key: string]: {
            action: string;
            channel: {
                $ref: string;
            };
            summary: string;
            traits?: {
                [key: string]: {
                    $ref: string;
                };
            };
            messages?: {
                [key: string]: {
                    $ref: string;
                };
            };
        };
    };
    components?: {
        messages?: {
            [key: string]: {
                name: string;
                title: string;
                summary: string;
                contentType: string;
                traits?: {
                    [key: string]: {
                        $ref: string;
                    };
                };
                payload?: {
                    $ref: string;
                };
            };
        };
        schemas?: {
            [key: string]: {
                type: string;
                properties: {
                    [key: string]: {
                        type: string;
                        minimum?: number;
                        maximum?: number;
                        description?: string;
                        enum?: string[];
                        format?: string;
                        items?: {
                            type: string;
                        };
                        $ref?: string;
                    };
                };
                description?: string;
            };
        };
        securitySchemes?: {
            [key: string]: {
                type: string;
                description: string;
            };
        };
        parameters?: {
            [key: string]: {
                description: string;
                schema: {
                    type: string;
                    format?: string;
                    minimum?: number;
                    maximum?: number;
                };
            };
        };
        messageTraits?: {
            [key: string]: {
                headers: {
                    type: string;
                    properties: {
                        [key: string]: {
                            type: string;
                            minimum?: number;
                            maximum?: number;
                        };
                    };
                };
            };
        };
        operationTraits?: {
            [key: string]: {
                bindings: {
                    kafka: {
                        clientId: {
                            type: string;
                            enum: string[];
                        };
                    };
                };
            };
        };
    };
}
function convertAsyncAPIToMermaid(asyncAPIDocument: AsyncAPIDocument): string {
  let mermaidCode = `flowchart TD\n`;

  // Add Streetlights Kafka API node
  mermaidCode += `    subgraph "${asyncAPIDocument.info.title}"\n`;

  // Add Servers subgraph
  mermaidCode += `        subgraph "Servers"\n`;
  Object.entries(asyncAPIDocument.servers).forEach(([serverName, serverInfo]) => {
      mermaidCode += `            ${serverName}["${serverName}"]\n`;
  });
  mermaidCode += `        end\n`;

  // Add Channels subgraph
  mermaidCode += `        subgraph "Channels"\n`;
  Object.entries(asyncAPIDocument.channels).forEach(([channelName, channelInfo]) => {
      mermaidCode += `            ${channelName}["${channelName}"]\n`;
  });
  mermaidCode += `        end\n`;

  // Add Operations subgraph
  mermaidCode += `        subgraph "Operations"\n`;
  Object.entries(asyncAPIDocument.operations).forEach(([operationName, operationInfo]) => {
      mermaidCode += `            ${operationName}["${operationName}"]\n`;
  });
  mermaidCode += `        end\n`;

  // Add Messages subgraph
  mermaidCode += `        subgraph "Messages"\n`;
  Object.entries(asyncAPIDocument.components.messages).forEach(([messageName, messageInfo]) => {
      mermaidCode += `            ${messageName}["${messageName}"]\n`;
  });
  mermaidCode += `        end\n`;

  mermaidCode += `    end\n`;

  // Add connections between servers and channels
  Object.entries(asyncAPIDocument.servers).forEach(([serverName]) => {
      Object.entries(asyncAPIDocument.channels).forEach(([channelName]) => {
          mermaidCode += `    ${serverName} --> ${channelName}\n`;
      });
  });

  // Add connections between channels and operations
  Object.entries(asyncAPIDocument.channels).forEach(([channelName, channelInfo]) => {
      Object.entries(asyncAPIDocument.operations).forEach(([operationName]) => {
          if (channelInfo.messages && channelInfo.messages[operationName]) {
              mermaidCode += `    ${channelName} --> ${operationName}\n`;
          }
      });
  });

  // Add connections between channels and messages
  Object.entries(asyncAPIDocument.channels).forEach(([channelName, channelInfo]) => {
      Object.entries(asyncAPIDocument.components.messages).forEach(([messageName]) => {
          if (channelInfo.messages && channelInfo.messages[messageName]) {
              mermaidCode += `    ${channelName} --> ${messageName}\n`;
          }
      });
  });

  // Add connections between operations and messages
  Object.entries(asyncAPIDocument.operations).forEach(([operationName, operationInfo]) => {
      Object.entries(asyncAPIDocument.components.messages).forEach(([messageName]) => {
          if (operationInfo.messages && operationInfo.messages[messageName]) {
              mermaidCode += `    ${operationName} --> ${messageName}\n`;
          }
      });
  });

  return mermaidCode;
}



try {
    const yamlFile = fs.readFileSync('asyncapi.yaml', 'utf8');
    const asyncAPIDocument: AsyncAPIDocument = yaml.load(yamlFile) as AsyncAPIDocument;
    const mermaidCode = convertAsyncAPIToMermaid(asyncAPIDocument);
    console.log(mermaidCode);
} catch (err) {
    console.error('Error......:', err);
}```

[ivangsa]

hey, you guys know how to use version control right?

you can start codiging as a team and pull requests to this new branch feature/mermaid-markdown I will merge your PRs withouth much review until you decide you got something working..

Thanks for all your effort and Happy Coding!!

[nikhilkalburgi]

Yes @ivangsa , I know and have already raised one to this project before. We can initially finalize the timeline and prototype together and implement it to build this feature.