airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

✨Source Asana: Migrate Python CDK to Low-code CDK

Open btkcodedev opened this issue 1 year ago • 19 comments

What

Migrating Source Asana to Low-Code CDK Closes airbytehq/airbyte-internal-issues#7057

How

Developed using (Configuration Based Source) low-code CDK

Recommended reading order

  1. spec.yaml
  2. manifest.yaml
  3. schemas/*

Tests

Integration & Acceptance
  • Test successfull

image

🚨 User Impact 🚨

  • Migration to low-code

btkcodedev avatar Apr 10 '24 06:04 btkcodedev

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 4, 2024 5:11pm

vercel[bot] avatar Apr 10 '24 06:04 vercel[bot]

Inline Schema ✅ Poetry lock file ✅ Breaking change mentioned in metadata ✅ Performed config migrations ✅ Added migrations.md docs file ✅ Docs updated ✅ Formatted :white_check_mark:

Test results: All pass image

btkcodedev avatar Apr 10 '24 06:04 btkcodedev

Marking PR as ready to merge

btkcodedev avatar Apr 10 '24 06:04 btkcodedev

Resolved comments

btkcodedev avatar Apr 15 '24 16:04 btkcodedev

Updated: Comments resolved (With fix for exposure of extra fields removed)

btkcodedev avatar Apr 17 '24 21:04 btkcodedev

Made the changes CI is passing :white_check_mark:

btkcodedev avatar Apr 19 '24 16:04 btkcodedev

@askarpets All comments resolved and CI is passing, Please continue merge process :bow:

btkcodedev avatar Apr 22 '24 15:04 btkcodedev

Oh sorry about that, will do

btkcodedev avatar Apr 23 '24 11:04 btkcodedev

Fixed

btkcodedev avatar Apr 24 '24 15:04 btkcodedev

The exposure problem is fixed, Please continue the merge process @askarpets image

btkcodedev avatar Apr 24 '24 15:04 btkcodedev

Resolved all comments CI green image

btkcodedev avatar Apr 25 '24 15:04 btkcodedev

@askarpets Any update here?

btkcodedev avatar May 09 '24 10:05 btkcodedev

Resolved comments and CI green

btkcodedev avatar May 16 '24 07:05 btkcodedev

@askarpets

btkcodedev avatar May 16 '24 07:05 btkcodedev

@askarpets, @lazebnyi Any updates here

btkcodedev avatar May 21 '24 17:05 btkcodedev

@askarpets, Does the opt field require a custom component? which declarative component inheritance would be opt for retrieving the response and modifying the request parameters?

btkcodedev avatar May 23 '24 17:05 btkcodedev

Any suggestion

btkcodedev avatar May 23 '24 17:05 btkcodedev

@askarpets, Does the opt field require a custom component? which declarative component inheritance would be opt for retrieving the response and modifying the request parameters?

I think you could just run the read command with --debug flag and get needed request params from the URLs and then add them to your manifest. Does that make sense?

askarpets avatar May 24 '24 09:05 askarpets

@askarpets But when looking at the implementation, it has something to do with schema right, Can you suggest anything for that 🙇 https://github.com/airbytehq/airbyte/pull/36951/files#diff-1a7657fc907cd4cc2de94c74edee1c1260b0e5549fd575880df07b27f19de530L66 Thanks again!

btkcodedev avatar May 24 '24 18:05 btkcodedev

@askarpets But when looking at the implementation, it has something to do with schema right, Can you suggest anything for that 🙇 https://github.com/airbytehq/airbyte/pull/36951/files#diff-1a7657fc907cd4cc2de94c74edee1c1260b0e5549fd575880df07b27f19de530L66 Thanks again!

It takes fields from schemas and converts them to opt_fields values. Since the schemas are static for this connector, you should be able to just hardcode this param. Otherwise, a custom requester should be implemented with overridden request_params method - please see example implementation.

askarpets avatar May 27 '24 08:05 askarpets

@askarpets, I've implemented the customRequester, but the problem is, Other request parameters are not being passed right now and substreams fail to fetch data. I've commented on the opt_field parameter line, but still, it doesn't pass, Could you please write your comments on the custom requester :pray: https://github.com/airbytehq/airbyte/pull/36951/files#diff-e10a6dd97d87fd5e5668385aaac9dfb0de57334b3ca29f6830ac4bcf038c430bR18

btkcodedev avatar May 31 '24 16:05 btkcodedev

@btkcodedev I made all needed changes to make it work.

askarpets avatar Jun 04 '24 13:06 askarpets

Thanks @askarpets :bow:

btkcodedev avatar Jun 04 '24 13:06 btkcodedev

All the comments are resolved I think, Please continue :pray: Thanks again!

btkcodedev avatar Jun 04 '24 13:06 btkcodedev