parabol icon indicating copy to clipboard operation
parabol copied to clipboard

Stop writing User to rethink

Open Dschoordsch opened this issue 3 years ago • 5 comments

Requires: #5384, #4867 and #5507

When all reads for User come from PG we should stop writing to rethink

Acceptance criteria

  • [ ] db.write and db.writeTable are removed
  • [ ] database/types/User.ts type is removed

Dschoordsch avatar Oct 05 '21 14:10 Dschoordsch

@Dschoordsch by all means feel free to throw away the db.write pattern & go directly to the database or a dataloader!

the db.write pattern of caching the User object was primarily because I wanted to see if I could remove some pressure from RethinkDB. Postgres can certainly handle the additional queries so we don't need to cache so aggressively.

mattkrick avatar Oct 05 '21 22:10 mattkrick

@Dschoordsch before we stop writing to rethink, I'd very much like to have hired our Data Architect and coordinate this making this change We have a bunch of queries in https://github.com/ParabolInc/parabol-data-notebooks we use to keep tabs on various low level metrics. It'd be great to:

  1. Get User moved over to a data warehouse or, failing that,
  2. Update these notebooks to query from Postgres instead

jordanh avatar Oct 12 '21 02:10 jordanh

Sure, I created #5507 so this is visible.

Dschoordsch avatar Oct 12 '21 05:10 Dschoordsch

Stale issue

github-actions[bot] avatar Jun 27 '22 00:06 github-actions[bot]

scrubbed, still a thing

mattkrick avatar Jul 11 '22 21:07 mattkrick

@tghanken can we stop writing to User in Rethink from your perspective?

Dschoordsch avatar Nov 08 '22 13:11 Dschoordsch

@tghanken can we stop writing to User in Rethink from your perspective?

Yes, there's nothing on our end stopping this at this point.

tghanken avatar Nov 08 '22 15:11 tghanken

@jordanh I still see this single User read in the mentioned notebooks https://github.com/ParabolInc/parabol-data-notebooks/blob/44152c7e57bf4c4952b959f958ac818838be3d36/Marketing/Users%20Last%20Seen.ipynb Is this used?

Dschoordsch avatar Nov 10 '22 14:11 Dschoordsch

I don't believe anyone is using those notebooks anymore. Everyone should now be reading from BigQuery instead. If necessary, I can submit a fix on that notebook code to move the read to Postgres, but I don't think it should stop us from moving forward with this issue.

tghanken avatar Nov 10 '22 16:11 tghanken

Stale issue

github-actions[bot] avatar May 10 '23 00:05 github-actions[bot]

fixed in #8116

mattkrick avatar May 10 '23 21:05 mattkrick