What

*Add capability to pull invoices data

How

*Add new stream invoices to pull the invoices based on start_date filter

🚨 User Impact 🚨

No user impact as this is not a breaking change

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

[x] Grant edit access to maintainers (instructions)
[ ] Secrets in the connector's spec are annotated with airbyte_secret
[x] Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
[ ] Code reviews completed
[x] Documentation updated
- [x] Connector's README.md
- [ ] Connector's bootstrap.md. See description and examples
- [ ] Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
[ ] PR name follows PR naming conventions

Feb 02 '23 16:02 ganpatagarwal

unit tests run

Results (4.97s):
      72 passed
Name                                                              Stmts   Miss  Cover   Missing
-----------------------------------------------------------------------------------------------
source_amazon_ads/__init__.py                                         2      0   100%
source_amazon_ads/constants.py                                        1      0   100%
source_amazon_ads/schemas/__init__.py                                 8      0   100%
source_amazon_ads/schemas/attribution_report.py                      21      0   100%
source_amazon_ads/schemas/common.py                                  51      1    98%   26
source_amazon_ads/schemas/invoices.py                               132      0   100%
source_amazon_ads/schemas/profile.py                                 16      0   100%
source_amazon_ads/schemas/sponsored_brands.py                        22      0   100%
source_amazon_ads/schemas/sponsored_display.py                       31      0   100%
source_amazon_ads/schemas/sponsored_products.py                      37      0   100%
source_amazon_ads/source.py                                          47      1    98%   134
source_amazon_ads/streams/__init__.py                                 8      0   100%
source_amazon_ads/streams/attribution_report.py                      81      0   100%
source_amazon_ads/streams/common.py                                  85      2    98%   96, 165
source_amazon_ads/streams/invoices.py                                87      4    95%   59-60, 117, 156
source_amazon_ads/streams/profiles.py                                21      0   100%
source_amazon_ads/streams/report_streams/__init__.py                  5      0   100%
source_amazon_ads/streams/report_streams/brands_report.py            10      0   100%
source_amazon_ads/streams/report_streams/brands_video_report.py      10      0   100%
source_amazon_ads/streams/report_streams/display_report.py           16      0   100%
source_amazon_ads/streams/report_streams/products_report.py          18      0   100%
source_amazon_ads/streams/report_streams/report_streams.py          240     19    92%   151, 204-205, 257-258, 377-378, 410-412, 417-418, 427-433
source_amazon_ads/streams/sponsored_brands.py                        26      0   100%
source_amazon_ads/streams/sponsored_display.py                       31      0   100%
source_amazon_ads/streams/sponsored_products.py                      41      0   100%
source_amazon_ads/utils.py                                           22      0   100%
unit_tests/__init__.py                                                0      0   100%
unit_tests/conftest.py                                               51      0   100%
unit_tests/test_attribution_report.py                                66      1    98%   47
unit_tests/test_invoices.py                                          54      1    98%   66
unit_tests/test_report_streams.py                                   297      0   100%
unit_tests/test_source.py                                            48      0   100%
unit_tests/test_streams.py                                          126      1    99%   77
unit_tests/test_utils.py                                             11      0   100%
unit_tests/utils.py                                                  32      0   100%
-----------------------------------------------------------------------------------------------
TOTAL                                                              1754     30    98%

Feb 02 '23 16:02 ganpatagarwal

acceptance test run

Test session starts (platform: darwin, Python 3.9.11, pytest 6.2.5, pytest-sugar 0.9.5) cachedir: .pytest_cache hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/sourc rootdir: /Users/ganpatagarwal/stoke/airbyte, configfile: pytest.ini plugins: sugar-0.9.5, requests-mock-1.9.3, mock-3.6.1, hypothesis-6.54.4, timeout-1.4.2, cov-3.0.0 collecting ... {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_config_match_spec[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_match_expected[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_docker_env[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_enum_usage[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_oneof_usage[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_required[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_optional[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_has_secret[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_secret_never_in_the_output[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_secret_is_properly_marked[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_property_type_is_not_array[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_object_not_empty[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_array_type[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_forbidden_complex_types[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_date_pattern[inputs0] ✓ airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_date_format[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_defined_refs_exist_in_json_spec_file[inputs0]

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_oauth_flow_parameters[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_backward_compatibility[inputs0] s

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestSpec.test_additional_properties_is_true[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestConnection.test_check[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestConnection.test_check[inputs1] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_discover[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_cursors_exist_in_schema[inputs0]

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_refs_exist_in_schema[inputs0]

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_keyword_exist_in_sch

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_defined_keyword_exist_in_sch

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_primary_keys_exist_in_schema[inputs0]

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_streams_has_sync_modes[inputs0] ✓

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_additional_properties_is_true[inputs0]

airbyte-integrations/bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py::TestDiscovery.test_backward_compatibility[inputs0] ✓ {"type": "LOG", "log": {"level": "WARN", "message": "The configured catalog was built with the discovered catalog from which the following ^C {"type": "LOG", "log": {"level": "INFO", "message": "/Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads

================================================================================== short test summary info ======================================== SKIPPED [1] ../../bases/connector-acceptance-test/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /Users/ganpatagarwal/.pyenv/versions/3.9.11/lib/python3.9/socket.py:704: KeyboardInterrupt (to show a full traceback on KeyboardInterrupt use --full-trace)

Results (31.04s): 30 passed 1 skipped

e-amazon-ads/.hypothesis/examples') -amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 3% ▍ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 5% ▌ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 8% ▊ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 11% █▏ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 13% █▍ 16% █▋ 18% █▉ 21% ██▏ 24% ██▍ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 26% ██▋ 29% ██▉ 32% ███▎ 34% ███▌ 37% ███▊ 39% ███▉ 42% ████▎ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ✓ 45% ████▌ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 47% ████▊ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 50% █████ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 53% █████▍ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 55% █████▌ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/integration_tests/invalid_config.json as configuration. It is the most recent version."}} 58% █████▊ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 61% ██████▏ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ✓ 63% ██████▍ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ✓ 66% ██████▋ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ema[inputs0-allOf] ✓ 68% ██████▉ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ema[inputs0-not] ✓ 71% ███████▏ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ✓ 74% ███████▍ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 76% ███████▋ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} ✓ 79% ███████▉ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} 82% ████████▎ {"type": "LOG", "log": {"level": "INFO", "message": "Using /Users/ganpatagarwal/stoke/airbyte/airbyte-integrations/connectors/source-amazon-ads/secrets/config.json as configuration. It is the most recent version."}} empty streams were removed: attribution_report_performance_adgroup, attribution_report_products, sponsored_brands_ad_groups, attribution_report_performance_campaign, sponsored_brands_report_stream, sponsored_brands_video_report_stream, sponsored_brands_campaigns, sponsored_display_report_stream, sponsored_products_report_stream, sponsored_brands_keywords, invoices, attribution_report_performance_creative."}} - Connector Acceptance Test run - bb16419dff64eae90ba6be72b53bd9cb26dbecf7 - PASSED"}} =========================================== are identical. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Feb 02 '23 16:02 ganpatagarwal

Thanks @ganpatagarwal we're going to review this contribution during the week

Apr 11 '23 20:04 marcosmarxm

/test connector=connectors/source-amazon-ads

:clock2: connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/4766928469 :white_check_mark: connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/4766928469 Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
source_amazon_ads/utils.py                                           22      0   100%
source_amazon_ads/streams/sponsored_products.py                      41      0   100%
source_amazon_ads/streams/sponsored_display.py                       31      0   100%
source_amazon_ads/streams/sponsored_brands.py                        26      0   100%
source_amazon_ads/streams/report_streams/products_report.py          37      0   100%
source_amazon_ads/streams/report_streams/display_report.py           18      0   100%
source_amazon_ads/streams/report_streams/brands_video_report.py      12      0   100%
source_amazon_ads/streams/report_streams/brands_report.py            12      0   100%
source_amazon_ads/streams/report_streams/__init__.py                  5      0   100%
source_amazon_ads/streams/profiles.py                                21      0   100%
source_amazon_ads/streams/attribution_report.py                      81      0   100%
source_amazon_ads/streams/__init__.py                                 8      0   100%
source_amazon_ads/schemas/sponsored_products.py                      37      0   100%
source_amazon_ads/schemas/sponsored_display.py                       31      0   100%
source_amazon_ads/schemas/sponsored_brands.py                        22      0   100%
source_amazon_ads/schemas/profile.py                                 16      0   100%
source_amazon_ads/schemas/invoices.py                               132      0   100%
source_amazon_ads/schemas/attribution_report.py                      21      0   100%
source_amazon_ads/schemas/__init__.py                                 8      0   100%
source_amazon_ads/constants.py                                        1      0   100%
source_amazon_ads/__init__.py                                         2      0   100%
source_amazon_ads/schemas/common.py                                  50      1    98%
source_amazon_ads/source.py                                          48      1    98%
source_amazon_ads/streams/common.py                                  85      2    98%
source_amazon_ads/streams/invoices.py                                90      4    96%
source_amazon_ads/streams/report_streams/report_streams.py          251     19    92%
-------------------------------------------------------------------------------------
TOTAL                                                              1108     27    98%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: can't populate stream because it requires real ad campaign
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================== 36 passed, 2 skipped in 285.21s (0:04:45) ===================

Apr 21 '23 16:04 marcosmarxm

@ganpatagarwal are you still willing to work on this? Please let us know or we will find another contributor, thanks!

Aug 15 '23 14:08 bmcgilli

Close due inactivity.

Sep 19 '23 16:09 marcosmarxm

airbyte
airbyte copied to clipboard

🎉 Source Amazon Ads : add new stream for invoices

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

unit tests run

acceptance test run

Build Passed

airbyte airbyte copied to clipboard

🎉 Source Amazon Ads : add new stream for invoices

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

unit tests run

acceptance test run

Build Passed

airbyte
airbyte copied to clipboard