pages-core icon indicating copy to clipboard operation
pages-core copied to clipboard

[WIP] Add localstack and cf-mock for local development

Open drewbo opened this issue 1 year ago • 2 comments

High-level PR intent

I'd like this PR to close #4326 by allowing all (or most) of the application functionality to be run locally in some capacity. There are some functions that are likely too difficult to replicate locally noted below.

Notes on Github

If AWS and CloudFoundry are mocked, the only "real" service that this will connect to locally is Github. I see two potential options on how to handle the Github connection:

  • Continue connecting to the "real" Github, knowing that certain functionality (e.g. build webhooks) cannot be replicated on a local application and will fail.
  • Develop a mock Github API to catch all requests. There does seem to be some prior art on this (like the wonderfully named Moctokit) but I didn't find anything immediately that would satisfy our need for a standalone server

Functionality Replication List (WIP)

  • [ ] ~Add site*~
  • [ ] ~Site build*~
  • [ ] ~Remove site*~
  • [x] Read archived build logs
  • [x] List published files/branches
  • [ ] ~Provision/delete domain~
  • [x] Read build task artifacts (includes presigned S3 URL)
  • [ ] ~Invite user~

*Require Github

Known Issues to write more about or solve

  • To work with localstack, our S3Client needs to be initialized with forcePathStyle: true. This is currently conditionally set via NODE_ENV but will break development because both the local application and the development environment use NODE_ENV=development. More info: https://docs.localstack.cloud/user-guide/integrations/sdks/javascript/
  • Localstack doesn't support persistence of data without a paid account. That's mostly okay because we can use the SDK to initialize our application with the necessary data. To do this, the localstack-setup service reads data from the local database to create the necessary buckets, development objects, etc. This generally works well with one exception: to create archived build logs, it actually deletes the corresponding items from the database, so on a new docker compose up, those items won't appear again without re-running docker compose run --rm app yarn create-dev-data
  • There is a potential (though not currently occurring) race condition in that the localstack-setup service requires both localstack and db for the wait-for-it.sh can only easily wait for one port to be available.
  • Localstack needs the Zscaler cert to be added in the local directory to work. Add documentation
  • Needs /etc/hosts change to resolve localstack host in browser. Add documentation

Changes proposed in this pull request:

security considerations

None, all services are running locally and don't expose external ports

drewbo avatar Dec 22 '23 20:12 drewbo