airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

Destination Mysql: DV2

Open edgao opened this issue 10 months ago • 2 comments

ported from https://github.com/airbytehq/airbyte/pull/34609 to use the kotlin cdk.

As a reminder, we're not doing any safe_cast stuff for mysql. I ran some manual tests in the perf test workspace; see Slack for summary.

We're generating (basically) the same data types as legacy normalization. The only diffs are numbers changing from float -> decimal(38, 9) (float only has 32-bit precision, so was causing problems) and timestamps changing from text -> varchar(1024).

Also we're adding new indexes - normalization was previously creating no indexes at all on the final table; we now have indexes on (extracted_at, pk, cursor), (extracted_at), and (raw_id). Legacy normalization and dv2 both index the raw table on raw_id; dv2 also indexes on loaded_at, extracted_at and extracted_at.

edgao avatar Apr 09 '24 17:04 edgao

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview May 3, 2024 8:30pm

vercel[bot] avatar Apr 09 '24 17:04 vercel[bot]

  • #36936 Graphite 👈
  • #36935 Graphite: 5 other dependent PRs (#34611 Graphite, #34612 Graphite, #34613 Graphite and 2 others)
  • #36929 Graphite
  • #36926 Graphite
  • master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @edgao and the rest of your teammates on Graphite Graphite

edgao avatar Apr 12 '24 18:04 edgao