botocore icon indicating copy to clipboard operation
botocore copied to clipboard

Update Response Parsing Protocol Tests

Open jonathan343 opened this issue 1 year ago • 1 comments

Note: Request serialization tests will be added in a follow up PR in effort to reduce the scope of this one.

Summary

This PR replaces the existing response parsing protocol tests in botocore/tests/unit/protocols/output with new tests generated from Smithy protocol test models. We use a version of these Smithy tests that are converted to the format currently supported by our existing test runner (test_protocols.py). This PR makes updates to the botocore Parsers and protocol test runner to comply with a majority of the new test cases.

Granular Protocol Ignore List

This PR introduces a protocol-tests-ignore-list.json file that can be used to ignore specific protocol test suites or cases. The structure for this list is defined below:

{
  "general": {
    "<test_type>": {
      "suites": [
        "Example test suite description",
      ],
      "cases": [
        "example-test-case-id"
      ]
    }
  },
  "protocols": {
    "<protocol_name>": {
      # Same as "general"  
    }
  }
}
  • <test_type> - There are two types of protocol tests.
    • input - Request serialization protocol test.
    • output - Response parsing protocol test.
    • Note: Only output will work until the request serialization tests are updated.
  • suites - A list of test suite descriptions to ignore. When specified, all test cases for the related suite will be ignored.
  • cases - A list of test case ids to ignore.
  • <protocol_name> - The protocol name as represented by its corresponding protocol test file name (without the .json) extension.
    • Supported values are ec2, json, json_1_0, query, rest-json, rest-xml. This may grow as we add more protocols.

jonathan343 avatar Sep 06 '24 15:09 jonathan343

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Please upload report for BASE (develop@b005187). Learn more about missing BASE report. Report is 204 commits behind head on develop.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #3247   +/-   ##
==========================================
  Coverage           ?   93.05%           
==========================================
  Files              ?       66           
  Lines              ?    14507           
  Branches           ?        0           
==========================================
  Hits               ?    13500           
  Misses             ?     1007           
  Partials           ?        0           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Sep 06 '24 15:09 codecov-commenter