airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

πŸ™ source-typeform: run up-to-date pipeline [2025-05-31]

Open airbyteio opened this issue 9 months ago β€’ 3 comments

Update source-typeform

This PR was autogenerated by running airbyte-ci connectors --name=source-typeform up_to_date --pull

We've set the auto-merge label on it, so it will be automatically merged if the CI pipelines pass. If you don't want to merge it automatically, please remove the auto-merge label. Please reach out to the Airbyte Connector Tooling team if you have any questions or concerns.

Operations

  • Upgrade the base image to the latest version in metadata.yaml: Successful

  • PATCH bump source-typeform version to 1.4.2: Successful

  • Build source-typeform docker image for platform(s) linux/amd64, linux/arm64: Successful

  • Get dependency updates: Successful

  • Create or update pull request on Airbyte repository: Successful

  • Add changelog entry: Successful

Dependency updates

We use syft to generate a SBOM for the latest connector version and the one from the PR. It allows us to spot the dependencies that have been updated at all levels and for all types of dependencies (system, python, java etc.). Here are the dependencies that have been updated compared to airbyte/source-typeform:latest. Keep in mind that :latest does not always match the connector code on the main branch. It is the latest released connector image when the head commit of this branch was created.

Type Name State Previous Version New Version
python CacheControl updated 0.13.1 0.14.2
python Jinja2 updated 3.1.4 3.1.6
python RapidFuzz updated 3.10.1 3.11.0
python airbyte-cdk updated 6.10.0 6.53.0
python airbyte_protocol_models_dataclasses updated 0.14.1 0.15.0
python anyio updated 4.7.0 4.9.0
python attrs updated 24.2.0 24.3.0
deb base-files updated 12.4+deb12u8 12.4+deb12u9
deb bsdutils updated 1:2.38.1-5+deb12u2 1:2.38.1-5+deb12u3
python build updated 0.10.0 1.2.2.post1
python cachetools updated 5.5.0 5.5.1
python certifi updated 2024.8.30 2024.12.14
python charset-normalizer updated 3.4.0 3.4.1
python click updated 8.1.7 8.1.8
python cryptography updated 43.0.3 44.0.2
python filelock updated 3.16.1 3.17.0
python h11 updated 0.14.0 0.16.0
python httpcore updated 1.0.7 1.0.9
python joblib updated 1.4.2 1.5.1
deb libblkid1 updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
deb libglib2.0-0 updated 2.74.6-2+deb12u4 2.74.6-2+deb12u5
deb libglib2.0-data updated 2.74.6-2+deb12u4 2.74.6-2+deb12u5
deb libmount1 updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
deb libopenjp2-7 updated 2.5.0-2 2.5.0-2+deb12u1
deb libsmartcols1 updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
deb libsystemd0 updated 252.31-1~deb12u1 252.33-1~deb12u1
deb libtiff6 updated 4.5.0-6+deb12u1 4.5.0-6+deb12u2
deb libudev1 updated 252.31-1~deb12u1 252.33-1~deb12u1
deb libuuid1 updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
python more-itertools updated 10.5.0 10.6.0
deb mount updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
python orjson updated 3.10.12 3.10.15
python pkginfo updated 1.11.2 1.12.0
python platformdirs updated 3.11.0 4.3.6
python poetry updated 1.6.1 1.8.4
python poetry-core updated 1.7.0 1.9.1
python poetry-plugin-export updated 1.6.0 1.8.0
python pydantic updated 2.10.3 2.10.5
python pydantic_core updated 2.27.1 2.27.2
binary python updated 3.10.14 3.11.11
python pytz updated 2024.1 2024.2
python serpyco-rs updated 1.11.0 1.13.0
python trove-classifiers updated 2024.10.21.16 2025.1.15.22
python typing_extensions updated 4.12.2 4.13.2
deb tzdata updated 2024a-0+deb12u1 2024b-0+deb12u1
python tzdata updated 2024.2 2025.1
python urllib3 updated 2.2.3 2.3.0
deb util-linux updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
deb util-linux-extra updated 2.38.1-5+deb12u2 2.38.1-5+deb12u3
python virtualenv updated 20.27.1 20.29.1
python wheel updated 0.44.0 0.45.1
python xmltodict updated 0.13.0 0.14.2
python Pygments added not present 2.19.1
python anyascii added not present 0.3.2
python boltons added not present 25.0.0
python fastjsonschema added not present 2.21.1
python google-api-core added not present 2.24.2
python google-auth added not present 2.39.0
python google-cloud-secret-manager added not present 2.23.3
python googleapis-common-protos added not present 1.70.0
python grpc-google-iam-v1 added not present 0.14.2
python grpcio added not present 1.71.0
python grpcio-status added not present 1.62.3
python markdown-it-py added not present 3.0.0
python mdurl added not present 0.1.2
python proto-plus added not present 1.26.1
python protobuf added not present 5.29.4
python pyasn1 added not present 0.6.1
python pyasn1_modules added not present 0.4.2
python rich added not present 13.9.4
python rich-click added not present 1.8.8
python rsa added not present 4.9.1
python whenever added not present 0.6.16
python exceptiongroup removed 1.2.2 not present
python pendulum removed 2.1.2 not present
python pytzdata removed 2020.1 not present
python tomli removed 2.1.0 not present

airbyteio avatar Feb 01 '25 21:02 airbyteio

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Feb 01 '25 21:02 CLAassistant

The latest updates on your projects. Learn more about Vercel for Git β†—οΈŽ

Name Status Preview Comments Updated (UTC)
airbyte-docs βœ… Ready (Inspect) Visit Preview πŸ’¬ Add feedback Jun 5, 2025 10:06pm

vercel[bot] avatar Feb 01 '25 21:02 vercel[bot]

source-typeform Connector Test Results

13 tests   13 βœ…β€ƒβ€ƒ19s ⏱️  1 suites   0 πŸ’€  1 files     0 ❌

Results for commit b561352e.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar May 10 '25 16:05 github-actions[bot]

/run-cat-tests

Connector CAT tests started... Check job output.

If the connector supports live tests or regression tests, you may want to run those now also:

CAT Tests completed. Important: Please check the job output to verify the results.

dbgold17 avatar Jun 04 '25 22:06 dbgold17

Acceptance test failure...

image

CAT Html report link

Looks like test_check is failing.

image

FAST tests seem to find the same thing:

image

https://github.com/airbytehq/airbyte/actions/runs/15454429012/job/43503755517?pr=53033

aaronsteers avatar Jun 05 '25 00:06 aaronsteers

@dbgold17 - While I understand that master is already failing, that does get drift vs what is published as the image. You could locally try something like docker run ... -v ... airbyte/source-typeform check --config=path/to/oauth_config.json

The challenge with that is just that you will need to get the syntax correct for volume-mapping the config from host to container volume.

aaronsteers avatar Jun 05 '25 00:06 aaronsteers

@dbgold17 - While I understand that master is already failing, that does get drift vs what is published as the image. You could locally try something like docker run ... -v ... airbyte/source-typeform check --config=path/to/oauth_config.json

The challenge with that is just that you will need to get the syntax correct for volume-mapping the config from host to container volume.

for my own understanding, why is there drift between running the integration tests using FAST on the master branch and running a check using the latest published connector image?

dbgold17 avatar Jun 05 '25 00:06 dbgold17

ok I did a bit of digging here @aaronsteers

Firstly, the command docker run -v $(pwd)/secrets/:/data/ airbyte/source-typeform:latest check --config /data/config_oauth.json fails with the same error that we see from integration tests and CATs

'Unable to connect to stream forms - 401 Client Error:  for url: https://api.typeform.com/oauth/token'

I checked out the credentials we use in that config and they appeared to be expired, so I went through the process of refreshing the access token, using the refresh token in the file, following the command described here. However, that gave me an error that the refresh token was invalid.

I went onto Airbyte Cloud and attempted to set up a typeform connection to see if I could get a new refresh token tied to our OAuth app. I noticed we only have one app and so it is probably shared between integration tests and our production app. Plugging that refresh token into the config and rerunning docker run -v $(pwd)/secrets/:/data/ airbyte/source-typeform:latest check --config /data/config_oauth.json gave me a successful check.

It seems that the process of refreshing tokens, also invalidates the old refresh token which is possible led to the issue in the first place. From their website:

NOTE: For security purposes, the refresh procedure will invalidate the old refresh token. Therefore, your application won't be able to use the old refresh token anymore. Instead, your application should use the newly generated refresh token returned in the response to this query.

next steps (let me know what you think)

  • I will store the new creds in the oauth_config on GSM, or wherever they live for fetching
  • re-run the tests on the PR
  • assuming they pass, we merge

dbgold17 avatar Jun 05 '25 04:06 dbgold17

@dbgold17 - This is great. Thanks for the update. And yes, next steps sound good.

It might also be worth checking if there's anything in 1password that might need to be updated, especially since you mention we might be using this for other purposes besides integration tests.

aaronsteers avatar Jun 05 '25 14:06 aaronsteers