generator

generator copied to clipboard

Current state

• https://github.com/asyncapi/generator/ that is a core
• https://github.com/asyncapi/generator-react-sdk for react render engine
• https://github.com/asyncapi/generator-filters/ with shared filters
• https://github.com/asyncapi/generator-hooks/ with shared hooks
• Multiple repositories with templates

This granularity is technically possible so templated development is flexible. But the fact that we make it possible for others, to easily create private, independent and use case specific templates doesn't mean we should have this level of granularity in asyncapi GH org

Proposal - discussion

How about we turn generator into a monorepo?

• generator-react-sdk - not a separate npm package, just natively part of generator
• generator-filters and generator-hooks also natively a part of generator (we still want to enable people to provide theirs independently)
• yes, templates as part of generator repo, but in case of templates, I guess we would need to publish separate packages 🤔 although I'm not 100% sure we need to

Why

• having to use generator-react-sdk as separate package is kinda confusing and in general my impression is that motivation for having it as a separate package was that then it can be written in TS
• over last 2 year we had zero improvements in hooks and filters, they are hidden small repos and template developers basically work on their own filters and hooks in their template repos.
• it would not be obligatory to move @asyncapi/*-template to the monorepo. But I think many would, as having it in our repo, with multiple maintainers/champions (yeah, we would have champion like setup like in modelina) would make it easier and quicker to reuse solutions from other templates, filters, components and other stuff. Under one repo it would be easier to focus or reusability and good standards (like always using modelina for models generation)

We would still be flexible, independent templates would be possible, hooks and filters too.

Having all under one umbrella would make generator a more vibrant project, more actively developed and visible.

wdyt?

any disadvantages that I do not see?

@derberg i saw we need to implement the monorepo for this i would like to work on this. Please let know how should i go about this.

@ayushnau for now it is proposal and a discussion

ok @derberg

@jonaslagoni @magicmatatjahu wdyt

Sounds good to me 🤙

Hey! would like to work on this issue, can i get it assigned? @derberg Thanks!

@ayushnau @Aryann15 are you folks still interested with helping here? did you have experience with introducing monorepo in the past?

first step would be project refactor and migration of first package from https://github.com/asyncapi/generator-filters

if you are interested working on it, please first share your plan before opening a PR

Hi I would love to work on this issue.

added to bounty:

• generator will be now a monorepo,
• main readme should still be focused on generator and there will just need to be a section where we explain that there are other projects included - short sentence intro and link to dedicated readme
• we need short development guide that will explain the setup of the repo
• filters are the first things we move
• filters repo will be archived
• we still release @asyncapi/generator and we still release @asyncapi/filters, just from one repo. Release can have the same number, we do not need separate versioning

any other questions?

Questions:

1. Is turborepo final for migrating generator repos to it.
2. So Generally we put all the functions and miscellaneous stuff like common scripts in. packages. (package containing all the shared stuff of all the apps.) which can be used so will we take some of shared stuff of the repos and put that into packages.
3. And do we want to put all the ci/cd flows in their own repos or will unite them into one.
4. We want to maintain git history of each app ?

@derberg

Bounty Issue's service comment

Text labels: bounty/2024-Q2, bounty/advanced, bounty/coding First assignment to third-party contributors: 2024-03-22 00:00:00 UTC+12:00 End Of Life: 2024-08-31 23:59:59 UTC-12:00

@asyncapi/bounty_team

Is turborepo final for migrating generator repos to it.

tbh I don't have any preference on what tool we choose. If we do npm workspaces, lerna or turborepo. Afaik you have experience with monorepo, so if you think turborepo is good investment, lets do it.

So Generally we put all the functions and miscellaneous stuff like common scripts in. packages. (package containing all the shared stuff of all the apps.) which can be used so will we take some of shared stuff of the repos and put that into packages.

we still need to release @asyncapi/generator and we will still have to release @asyncapi/filters. If there will be some common code then definitely we should have shared commons.

Filters technically are a feature only for generator templates that run on Nunjucks engine. So template developer uses it in the way that they add filters package to dependencies and then adds module name to filters in the config. So in theory, your job is a simple as move filters in, make sure they are released and that is it. But would be super nice if you can come up with an idea how to make sure @asyncapi/filters are included in asyncapi/generator by default, so nobody have to enable them by configuration.

And do we want to put all the ci/cd flows in their own repos or will unite them into one.

afaik workflows from https://github.com/asyncapi/generator-filters can be dropped as they are all anyway pulled from .github repo (so are the same that also in generator) repo. You just need to make sure that whatever workflows we have here, for release or bump - will still work

We want to maintain git history of each app

Not a strong requirement - just lemme know what options we have. The plan is that basically this https://github.com/asyncapi/generator-filters will be archived and moved to https://github.com/asyncapi-archived-repos so history will still be there. But if there is some cool git magic available so we can preserve history of filters in generator, lets see. I admit I'm not such a git expert

---

Please also work closely with people from https://github.com/asyncapi/parser-js/issues/963 as might be that best would be to have the same solution in both

@AyushNautiyalDeveloper so, still interested?

yeah Sure @derberg once you answer those question I will add short development guide. as required.

I think we have twins here 😄

btw, my answers are above

@ayushnau ok, please go ahead, the issue is yours

ok thanks will start working on this then.

Bounty Issue's Timeline

Complexity Level	Assignment date (by GitHub)	Start date (by BP rules)	End date (by BP rules)	Draft PR submission	Final PR submission	Final PR merge
Advanced	2024-03-21	2024-04-01	2024-05-24	2024-04-19	2024-05-10	2024-05-24

Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.

Keep in mind the responsibility for violations of the Timeline.

Comparison of Repository Management Tools

1. So I have Decided to go with Turborepo. Here is a comparision for different options we had. though Nx was also good.

Feature	TurboRepo	Lerna	Yarn Workspaces	Nx
Performance	High performance with caching and parallelization	Decent performance, may slow down with large repos	Decent performance, can suffer with large repos	High performance with advanced caching and builders
Dependency Management	Integrated dependency resolution	Basic dependency management	Integrated with Yarn, follows Yarn's resolution	Integrated dependency graph for efficient builds
Versioning	Integrated version management	Incremental versioning with fixed or independent	Integrated with Yarn, follows SemVer	Semantic versioning with versioning strategies
Language Support	Primarily JavaScript and TypeScript	Primarily JavaScript and TypeScript	JavaScript, TypeScript, and more	JavaScript, TypeScript, and more
Community Support	Growing community support	Large and established community	Strong community support	Strong community support
Ease of Setup	Straightforward setup and configuration	Moderate setup with configuration options	Integrated with Yarn, relatively easy to set up	Straightforward setup and integration
Integration with CI/CD	Supports various CI/CD pipelines	Compatible with common CI/CD tools	Compatible with common CI/CD tools	Integrated CI/CD support and workflows

Adding a single repo will look like this

1. Install and do basic setup of TurboRepo. Note: We will be using npm as a package manager as we have used npm in every package.

2. Add the @asyncapi/filter code to the apps directory of the turborepo.

3. Add the configuration for the @asyncapi/filter to the Turbo.json file in the root directory.

4. Add all the required scripts in the package.json file for the @asyncapi/filter. (To run apps package.json scripts from root)

Question:

1. What should be the name of the repo? It can't be "Generator" as we are going to use it for its own repo.

https://github.com/ayushnau/generator

Added the @asyncapi/generator-filters repo

And For git history It is possible to maintain the history of every repo merged into one. As i have done with this current repo. @derberg

can you specify what is the goal of https://github.com/ayushnau/generator ? just for testing? The changes that you introduce should be done in a typical way through a fork

What should be the name of the repo? It can't be "Generator" as we are going to use it for its own repo.

we keep the name for now, is that a blocker? all additional stuff we will move to generator are considered plugins/extensions - apps is probably a good name too

So I have Decided to go with Turborepo. Here is a comparision for different options we had

thanks for great comparizon. Turborepo indeed looks interesting. Also notices even nodejs started using it https://github.com/nodejs/nodejs.org

1. The purpose of that repository is solely for testing and demonstration purposes.Other apps will also be included inside the apps directory (including the generator itself.)
2. No, it is not a blocker, just asking for clarity.
3. 👍

awesome, feel free to DM me whenever you need help or there is something new I should look at - I'm a bit under water with notifications, so DM is working very well for me atm

https://github.com/asyncapi/generator/pull/1155

@ayushnau I noticed you closed the PR, all good? what's the plan?

@derberg I am getting an issue with running test command. This is coming due to the test trying to access the nims from node_modules. but turborepo have installed it in base node_modules. where all the internal and shared dependencies packages are installed.

Solution: I think what we can do is just change the directory

 '^nimma/legacy$': '<rootDir>/node_modules/nimma/dist/legacy/cjs/index.js',
    '^nimma/(.*)': '<rootDir>/node_modules/nimma/dist/cjs/$1', 

to correct directory.

yup, go ahead 😀

@ayushnau, please provide an update to the PR.