design: Monorepo Refactor -> Multiple Repositories

[yourbuddyconner]

We currently live in package dependency hell -- the monorepo's property that all code is co-located in the same filesystem ended up being a little bit of a crutch and we made some bad code hygiene decisions out of expediency.

We're going to claw ourselves out of Dante's Typescript Inferno via Yarn Workspaces, adding some enforced rules on dependencies and the relationships between typescript packages to make things intuitive and easy.

Goal:

• Everything Typescript is a "workspace"
• This should set us up to split up the monorepo into smaller pieces

Sequencing:

We're going to refactor nomad-monorepo into a series of smaller monorepo work trees.

• nomad-config -> Configuration root, most releasable packages depend on this (this will be a git submodule on most of the other repos below that depend on it)
• nomad-sol -> Contracts, depends on nomad-config
• nomad-rs -> Rust folder, same structure as current, depends on nomad-sol rust bindings and nomad-config
• nomad-ts -> TS libraries and services, depends on nomad-sol ABIs, nomad-config (everything here would be a deployable package on npm)
• nomad-services -> for tools such as nomad-indexer, nomad-monitor, keymaster, local-environment

How do we get here?

In this order:

1. Disable solidity tests
2. Create unified config
    - Parsers in rs/ts
3. Refactor rust & TS to use unified config
    - rust nomad-base config loader
    - ts sdk
    - ts deploy process
4. Split up repos
5. Make sure everything still works

Post-move to new repo structure

Not in any order, doesn't need to happen immediately

- Split up sdk into multiple packages
- Split up nomad-xapps into multiple packages
- Rewrite readmes
- Set up new CI in each new repo

Notable Changes:

• Combine and flatten thesolidity and typescript/typechain packages into multiple contracts-* packages
  • contracts-core - Core contract source code and typescript interfaces
  • contracts-bridge - Bridge contract source code and typescript interfaces
• Addition of an examples workspace, containing sub-workspaces for individual examples
• rust workspace with sub-workspaces containing individual Rust crates

Outstanding Questions:

• How should we avoid a potential circular dependency with nomad-deploy (one potential solution is moving to nomad-services, we could also split out multi-provider from sdk, TBD here)
• Unit tests work just fine in their respective repos, but how should integration tests be set up since these are across repos?
• Should we individually package the suite of SDK packages or should we pack them into a mega @nomad-xyz/sdk package and namespace functionality via sub-directories (ex: @nomad-xyz/sdk/core or @nomad-xyz/sdk/bridge)?
• Same question for the suite of contracts packages @nomad-xyz/contracts/core vs @nomad-xyz/contracts-core
• Should we un-pack nomad-tests and put tests for individual contract suites in their own packages? (core, bridge, etc)
• Should we do the same with nomad-deploy?

.
├── nomad-sol/
│   ├── config (submodule -> nomad-config)
│   └── packages/
│       ├── core/
│       │   ├── typechain (aka @nomad-xyz/contract-interfaces)
│       │   └── contract-code 
│       ├── xapp-base
│       ├── bridge
│       ├── example-xapp
│       └── deploy (depends on @nomad-xyz/contract-interfaces and @nomad-xyz/sdk)
├── nomad-config/
│   ├── development
│   ├── staging
│   └── production 
├── nomad-rs/
│   ├── config (submodule -> nomad-config)
│   └── packages/
│       ├── agents
│       ├── chains
│       ├── nomad-base
│       ├── nomad-core
│       ├── nomad-test
│       └── tools
├── nomad-ts/
│   ├── config (submodule -> nomad-config)
│   └── packages/
│       ├── multi-provider
│       ├── sdk
│       ├── bridge
│       ├── govern
│       └── example-ui
└── nomad-services/
    ├── config (submodule -> nomad-config)
    └── packages/
        ├── nomad-indexer
        ├── nomad-monitor 
        ├── keymaster 
        └── local-environment

What do we do when we add a new network?

Say we're adding a new network to the dev environment, what happens now?

• deploy contracts
• update dev config with new addresses
• update all the other packages that depend on the config (there should be no circular dependencies)

[mhluongo]

Reading this and I'm excited for you :grin:

[yourbuddyconner]

Wow, it has been such a journey between the time I wrote this and now 🤩