boulder icon indicating copy to clipboard operation
boulder copied to clipboard

Remove use of MultiInserter's "RETURNING" functionality

Open aarongable opened this issue 1 year ago • 2 comments

This doesn't work in MySQL, so we need to remove it before we migrate.

We may want to replace it with random ID generation rather than auto-increment, if we're worried about the efficiency of inserting 100 rows for a maximally-sized newOrder request.

aarongable avatar Sep 23 '24 21:09 aarongable

Note that replacing the Auto-Increment behavior will break the current partition management scheme.

jcjones avatar Sep 23 '24 22:09 jcjones

Step 1 here is:

  • Add a feature flag to control this behavior
  • Behind that flag, replace multi-inserter with a bunch of (parallel) single inserts
  • Use the SA's existing "parallelismPerRPC" config to control the parallelism here

We also probably want to reduce the configured parallelism per RPC, as we believe that may have been a contributing factor in the SA<->ProxySQL connection churn we've historically had problems with.

From there, we can flip the flag in staging/prod, and easily turn it off if it causes problems. If it doesn't cause problems, we're good here -- if it does cause problems, we can dive into batch inserts, random ID generation, and all the knock-on effects of that.

aarongable avatar Sep 24 '24 18:09 aarongable