data-diff

data-diff copied to clipboard

For row-based databases if indexes are missing on the columns queried, the queries will be substantially slower on large tables (unless you're querying all columns).

Today, you can run with --interactive to see an EXPLAIN before each query. In --debug you can see each query, to run the EXPLAIN yourself.

However, we don't do anything beyond that. We should spit out at a warning logging level if we suspect we might be issuing queries that aren't optimal, due to a lack of indexes. Either we can check the schema that we have all the necessary indexes, or we can try to parse the EXPLAIN. I'm in favour of the schema, it's a little less real, but it means less maintenance per driver 😅

Of course, compound indexes will be better, but a bunch of individual may be good enough on many databases that can scan multiple in parallel (an assumption worth verifying with at least Postgres and MySQL).