Restructure repo into Monorepo

[brent-hoover]

Restructure this repo so that all packages (including library packages like Logger and Error) are all contained in this repo and published upon merge of PR.

This will do a lot to make this easier to manage, understand, and maintain.

The main challenge here is to incorporate the existing workflow of using Semantic Release along with auto-publishing packages.

Definition of Done:

1. All Reaction packages (including lib plugins like Logger and Error) live inside a folder called "packages"
2. Whenever a PR is merged into a package a new version gets published via semantic release exactly like it does now
3. Whenever a plugin is updated, a new version of reaction is released semantically unless the package bump is breaking (e.g. a MAJOR change under semantic release)
4. The build time (to run reaction) doesn't increase(at-least significantly).

Background info:

1. An analysis of the different options available is located here
2. Some feedback from @aldeed

I agree that a mono repo can be nice for this case, and I also think you want to avoid incrementing and publishing all packages when only some have changed. I’ve struggled with Lerna enough to not love it as a solution.

I’d probably lean toward a bash or node script custom for what needs to happen, with each package folder managing its own scripts.
I am not sure exactly what it would be on CircleCI, but on GitHub Workflows I have had success with this config:
jobs:
  project-changed:
   runs-on: ubuntu-latest
   outputs:
    didChange: ${{ steps.projectDiff.outputs.count > 0 }}
   steps:
   - uses: actions/checkout@v2
   - name: Determine whether package files changed
     id: projectDiff
     uses: technote-space/get-diff-action@v6
     with:
      PATTERNS: |
      api-plugin-whatever/**

  publish:
   runs-on: ubuntu-latest
   needs: project-changed
   if: ${{ needs.project-changed.outputs.didChange == 'true' }}

You would need a separate workflow file for each package, but the “publish” job (or any job that you make dependent on the “project-changed” check) would not run. So that would limit it to only running checks and publishing packages with changed files.

That is using this: https://github.com/technote-space/get-diff-action

Then the actual `publish` job would do something similar to what each repo does for the publish step right now, using semantic-release to determine the version and publish it.
As for being able to figure out which commits go with which packages for the semantic version bump, have you looked at this? https://www.npmjs.com/package/semantic-release-monorepo

[Akarshit]

I got the KT for this from Alvaro. Have started working on this.

[Akarshit]

I did the initial investigation into this. But couldn't find a ready made solution that's production ready. For now the work on this has been paused.

[janus-reith]

I'm wondering, since the node API works in ESM mode, and all of the api modules are distributed as ESM too, could the API modules list peerdependencies instead, and could the remaining packages like Logger and Error be migrated to ESM too? My intention would be to make individual per-package build steps obsolete that way, and lerna might not be necessary anymore then, atleast for the purpose of development? And next up, we might even skip using NPM altogether and reference the required packages right from the github repo.

I'm afraid lerna might cause more slowdown, and a solution to this could be more lightweight than that, since overall the goal would be to cut down on complexity.

Adding Typescript as it was also researched recently could become another painpoint though, since right now TSC and SWC with node in ESM mode don't seem to have a common understanding which file extension should be present. ATM I would consider though to diverge from the current TS standards, and take the Deno approach, which would be to simply import plain TS as ESM and pass it to SWC. Same can be accomplished in node using a Loader.

[Akarshit]

Resuming the work on this one. To clarify this is how the monorepo is expected to work:

1. All the api-plugins-* live inside a folder called "plugins"
2. Whenever a plugin is updated, a new version of reaction is released sematically.
3. The history of commits for all individual plugins is merged together into the history of commits for reaction.
4. The build time (to run reaction) doesn't increase(at-least significantly).

@zenweasel Is there anything I missed in this?

[brent-hoover]

The plugin should get bumped/published and reaction should get bumped on minor/patch versions

[brent-hoover]

This should be all reaction packages including ones that aren't plugins like Logger and Error