airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

🎉 Source Okta: added parameter 'start_date'

Open drrest opened this issue 2 years ago • 4 comments

What

added: parameter 'start_date' to source Okta

How

Added filed in request parameters.

Recommended reading order

  1. source.py
  2. spec.json

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • [ ] Create a non-forked branch based on this PR and test the below items on it
  • [ ] Build is successful
  • [ ] If new credentials are required for use in CI, add them to GSM. Instructions.
  • [ ] /test connector=connectors/<name> command is passing
  • [ ] New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

drrest avatar Jul 26 '22 21:07 drrest

/test connector=connectors/source-okta

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742745203 :x: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742745203 :bug: https://gradle.com/s/u7ca3a7gwphy4

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
=================== 1 failed, 30 passed in 82.51s (0:01:22) ====================

drrest avatar Jul 26 '22 21:07 drrest

/test connector=connectors/source-okta

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742864137 :x: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742864137 :bug: https://gradle.com/s/olm6wpxnbnxn2

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
=================== 1 failed, 30 passed in 86.55s (0:01:26) ====================

drrest avatar Jul 26 '22 22:07 drrest

/test connector=connectors/source-okta

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2760467751 :white_check_mark: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2760467751 Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        77      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/tests/test_incremental.py       121     25    79%
source_acceptance_test/utils/common.py                  77     17    78%
source_acceptance_test/tests/test_core.py              307    106    65%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
------------------------------------------------------------------------
TOTAL                                                  970    246    75%
Name                      Stmts   Miss  Cover
---------------------------------------------
source_okta/__init__.py       2      0   100%
source_okta/source.py       189      1    99%
---------------------------------------------
TOTAL                       191      1    99%

Build Passed

Test summary info:

All Passed

drrest avatar Jul 29 '22 11:07 drrest

Please make ./gradlew clean format after all your work has been done and commit those changes as well because this CI job fails due to bad formatting, optionally you can merge with master to pick up latest changes before merging into master.

bazarnov avatar Jul 30 '22 18:07 bazarnov

/publish connector=connectors/source-okta

:clock2: Publishing the following connectors:
connectors/source-okta
https://github.com/airbytehq/airbyte/actions/runs/2791973155

Connector Did it publish? Were definitions generated?
connectors/source-okta :x: :x:

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

drrest avatar Aug 03 '22 19:08 drrest

/publish connector=connectors/source-okta

:clock2: Publishing the following connectors:
connectors/source-okta
https://github.com/airbytehq/airbyte/actions/runs/2792089523

Connector Did it publish? Were definitions generated?
connectors/source-okta :x: :x:

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

drrest avatar Aug 03 '22 19:08 drrest

/test connector=connectors/source-okta

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806709455 :x: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806709455 :bug: https://gradle.com/s/iwjfobjxmuima

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_token_authentication_init
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_oauth2_authentication_init
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_oauth2_authentication_wrong_credentials_record
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_oauth2_authentication_wrong_oauth_config_bad_auth_type
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_connection_ok
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_connection_error_status_code
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_connection_error_with_exception
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_streams - At...
	 FAILED unit_tests/test_source.py::TestAuthentication::test_oauth2_refresh_token_ok
	 FAILED unit_tests/test_source.py::TestAuthentication::test_oauth2_refresh_token_failed
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.OK-False]
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.BAD_REQUEST-False]
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.TOO_MANY_REQUESTS-True]
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.INTERNAL_SERVER_ERROR-True]
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_request_params
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_parse_response
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_backoff_time
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_request_params
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_incremental_okta_stream_parse_response
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_incremental_okta_stream_backoff_time
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_backoff_time_empty
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_back_off_now
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_get_updated_state
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_http_method
	 FAILED unit_tests/test_streams.py::TestNextPageToken::test_next_page_token - ...
	 FAILED unit_tests/test_streams.py::TestNextPageToken::test_next_page_token_empty_params
	 FAILED unit_tests/test_streams.py::TestNextPageToken::test_next_page_token_link_have_self_and_equal_next
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_stream_users - TypeE...
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_request_params_out_of_next_page_token
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_source_request_params_have_next_cursor
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_source_request_params_have_latest_entry
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_source_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamCustomRoles::test_custom_roles
	 FAILED unit_tests/test_streams.py::TestStreamCustomRoles::test_custom_roles_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamPermissions::test_permissions - ...
	 FAILED unit_tests/test_streams.py::TestStreamPermissions::test_permissions_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroups::test_groups - TypeError:...
	 FAILED unit_tests/test_streams.py::TestStreamGroups::test_groups_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members_request_params_with_latest_entry
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members_slice_stream
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_member_request_get_update_state
	 FAILED unit_tests/test_streams.py::TestStreamGroupRoleAssignment::test_group_role_assignments
	 FAILED unit_tests/test_streams.py::TestStreamGroupRoleAssignment::test_group_role_assignments_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroupRoleAssignment::test_group_role_assignments_slice_stream
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs - TypeError: __i...
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs_request_params_for_since
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs_request_params_for_until
	 FAILED unit_tests/test_streams.py::TestStreamUserRoleAssignment::test_user_role_assignments
	 FAILED unit_tests/test_streams.py::TestStreamUserRoleAssignment::test_user_role_assignments_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamUserRoleAssignment::test_user_role_assignments_slice_stream
	 FAILED unit_tests/test_utils.py::test_get_start_date[config2-expected_date2-expected_messages2]
	 [31m========================= [31m[1m54 failed[0m, [32m2 passed[0m[31m in 1.51s[0m[31m =========================[0m

lazebnyi avatar Aug 05 '22 23:08 lazebnyi

/test connector=connectors/source-okta

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515 :x: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515 :bug: https://gradle.com/s/oeakgteownrkm

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AssertionError: All ...
FAILED test_core.py::TestBasicRead::test_read[inputs1] - AssertionError: All ...
=================== 2 failed, 34 passed in 89.60s (0:01:29) ====================

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515 :x: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515 :bug: https://gradle.com/s/ok52jqgzbe6gm

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs1] - AssertionError: All ...
FAILED test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
=================== 2 failed, 34 passed in 220.11s (0:03:40) ===================

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515 :x: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515 :bug: https://gradle.com/s/ysy636cvvkr7c

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_utils.py::test_get_start_date[config1-expected_date1-expected_messages1]
	 FAILED unit_tests/test_utils.py::test_get_start_date[config2-expected_date2-expected_messages2]
	 [31m================== [31m[1m2 failed[0m, [32m54 passed[0m, [33m132 warnings[0m[31m in 0.71s[0m[31m ==================[0m

lazebnyi avatar Aug 05 '22 23:08 lazebnyi

/test connector=connectors/source-okta

:clock2: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2807174148 :white_check_mark: connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2807174148 Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        82      6    93%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/tests/test_incremental.py       121     25    79%
source_acceptance_test/utils/common.py                  77     17    78%
source_acceptance_test/tests/test_core.py              355    107    70%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
------------------------------------------------------------------------
TOTAL                                                 1023    247    76%
Name                           Stmts   Miss  Cover
--------------------------------------------------
source_okta/utils.py              44      0   100%
source_okta/authenticator.py      14      0   100%
source_okta/__init__.py            2      0   100%
source_okta/source.py            174      7    96%
--------------------------------------------------
TOTAL                            234      7    97%

Build Passed

Test summary info:

All Passed

lazebnyi avatar Aug 06 '22 01:08 lazebnyi

/publish connector=connectors/source-okta

:clock2: Publishing the following connectors:
connectors/source-okta
https://github.com/airbytehq/airbyte/actions/runs/2816679958

Connector Did it publish? Were definitions generated?
connectors/source-okta :white_check_mark: :white_check_mark:

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

lazebnyi avatar Aug 08 '22 08:08 lazebnyi