Backend repo migration

[dorranh]

Hi EdNutting and @rossng, I've been looking into migrating clowdr-backend to this repository and had a quick question. I just wanted to confirm that you would like to combine the back-end components defined in clowdr-backend with the ones defined here in backend/cloud.

If so, I was thinking to do this as a two step process where we 1. copy the back-end to this repo but keep it as a separate library then 2. combine the newly added components with those in backend/cloud. This way we can validate that everything still works after the move before starting with any refactoring. Let me know if that sounds good to you all, and I can go ahead and prepare a PR.

Plan:

• [ ] (#271) Copy clowdr-backend to this repository
  • [ ] Validate that builds, etc. continue to run as intended. Heroku app / build script will need to be updated since clowdr-backend now lives in a subdirectory.
• [ ] Mark old clowdr-backend repo as deprecated
• [ ] Copy clowdr-db-schema to this repository
• [ ] Update source path for clowdr-db-schema in various clowdr-web-app packaging

[EdNutting]

Hi Dorran,

I was hoping we would keep the two separate in separate folders for now. One is the Parse backend, the other (currently clowdr-backend) runs on Heroku. There is some purpose to this: Parse running on Back4App has a very low request rate limit. So having the Twilio callbacks come through Heroku helps spread the load.

The most important thing here is to get everything into a monorepo so clowdr-db-schema is integrated and the code shared between the frontend and the Twilio backend (clowdr-backend). The Twilio backend relies only on the type information (iirc) not the functions. This will significantly reduce the overhead of making changes to the db schema / to the data-layer/cache code.

The Parse backend is entirely in JS at the moment and upgrading it to TS may be wasted effort given our plans to shift to a different backend (and replace both existing backends).

So yes to step 1 only.

[dorranh]

Hi @EdNutting, thanks for the additional info! That makes sense, I wasn't aware of the request rate limit of Back4App. One other thing to keep in mind for migrating clowdr-db-schema is that if we simply copy the schema library into this repo, the git history of its previous versions won't come with it. This isn't a huge issue since we can flag the old clowdr-db-schema repo as deprecated without deleting it, but it may slightly complicate migration scripts, etc. if you have any existing databases which are using an older schema version (since they will need to load the schema from the old repo and not this one).

If backwards compatibility isn't a big concern then IMO it should be fine to simply copy the schema library here, otherwise we can also use a git submodules (I think I had asked about this regarding clowdr-backend but not for the schemas).

[EdNutting]

Just copy across - there's nothing in the way of db migrations (though some scripts will have to be directed at the right folder rather than the NPM package). Git history is entirely me/Ross at this stage so doesn't matter.

The challenge with copying the folder across is getting the webpack and heroku builds to both accept it. I am entirely unsure how that will work.