airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

🚨🚨🎉Source Pinterest: migrate to low-code

Open askarpets opened this issue 1 year ago • 6 comments

What

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/7435

How

Migrated streams to low-code

Review guide

  1. manifest.yaml
  2. source.py

User Impact

Format of state has changed for incremental streams

Can this PR be safely reverted and rolled back?

  • [ ] YES 💚
  • [x] NO ❌

askarpets avatar Apr 30 '24 15:04 askarpets

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 24, 2024 8:46am

vercel[bot] avatar Apr 30 '24 15:04 vercel[bot]

@brianjlai here is one of runs: https://github.com/airbytehq/airbyte/actions/runs/9211964690 - the only failed test is related to discover, however, I don't see any issues when comparing original and new schemas

askarpets avatar May 23 '24 18:05 askarpets

@brianjlai here is one of runs: https://github.com/airbytehq/airbyte/actions/runs/9211964690 - the only failed test is related to discover, however, I don't see any issues when comparing original and new schemas

I know this was a breaking change, but was this expected? When you say you don't see any issues, do you mean that the diffs on the control vs. yours do not differ at all? Or that the deviations don't change the behavior?

I think I want to understand why the catalog changed and we should be sure that was intended. If you run discover() against master and your version, what is the difference between the two catalogs?

brianjlai avatar May 23 '24 20:05 brianjlai

@brianjlai here is one of runs: https://github.com/airbytehq/airbyte/actions/runs/9211964690 - the only failed test is related to discover, however, I don't see any issues when comparing original and new schemas

I know this was a breaking change, but was this expected? When you say you don't see any issues, do you mean that the diffs on the control vs. yours do not differ at all? Or that the deviations don't change the behavior?

I think I want to understand why the catalog changed and we should be sure that was intended. If you run discover() against master and your version, what is the difference between the two catalogs?

I double-checked the catalogs manually - the schemas are completely the same except the order of streams in the catalogs. This is because I used different order in the manifest file, however, I don't think this should cause any issues. Additionally, I've created a connection using our test account since there are more streams with data than I could find in customers' connections and ran the tests with this connection - please see the result.

askarpets avatar May 24 '24 09:05 askarpets

Hey @brianjlai & @askarpets, we definitely don't intend to be checking the stream order. I actually thought we'd fixed this. Will look into it, and sorry for the confusion!

clnoll avatar May 24 '24 10:05 clnoll

Hey @brianjlai & @askarpets, we definitely don't intend to be checking the stream order. I actually thought we'd fixed this. Will look into it, and sorry for the confusion!

Thank you @clnoll!

askarpets avatar May 24 '24 10:05 askarpets