aws-sdk-ruby icon indicating copy to clipboard operation
aws-sdk-ruby copied to clipboard

[DRAFT] Update protocol tests

Open jterapin opened this issue 4 months ago • 5 comments

Objectives

  • Replace existing protocol-tests with Smithy's definition of protocol-tests
  • Update the test runner to consume these accordingly (and clean up)
  • Resolve test failures
  • Implement a mechanism to skip test cases (and have a log of skipped cases with reasons why)
  • Carry over protocol-tests that was not covered by Smithy's definition (i.e. eventstreams)
  • Update existing protocol-tests (api-gateway, carry overs) to include descriptive format

Current Log

  • aws-sdk-code-generator
    • Added short as ruby shape
  • aws-sdk-core
    • Allow nil value in the following shapes: StringShape, IntegerShape and BooleanShape
    • Added Content-Type header for rest-xml requests
    • For rest-json and rest-xml, a specific Content-Type is set when the payload shape is either BlobShape or StringShape
    • Fix deserializing xml to produce an empty string for empty self-closed blobs
    • Fix xml parsing for xml namespace

Starting total of failures: 349


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

  1. To make sure we include your contribution in the release notes, please make sure to add description entry for your changes in the "unreleased changes" section of the CHANGELOG.md file (at corresponding gem). For the description entry, please make sure it lives in one line and starts with Feature or Issue in the correct format.

  2. For generated code changes, please checkout below instructions first: https://github.com/aws/aws-sdk-ruby/blob/version-3/CONTRIBUTING.md

Thank you for your contribution!

jterapin avatar Feb 23 '24 16:02 jterapin

You have made a change to core without a corresponding change to the CHANGELOG.md. This change will not result in a new version and will not published unless an entry is added to CHANGELOG.md

github-actions[bot] avatar Feb 27 '24 15:02 github-actions[bot]

Current total of failures: 252

jterapin avatar Feb 27 '24 15:02 jterapin

Current status: 197 failures, 50 ignored

jterapin avatar Feb 28 '24 15:02 jterapin

Current status: 160 failures, 51 ignored

jterapin avatar Mar 01 '24 18:03 jterapin

Current failures: 57

jterapin avatar Mar 04 '24 21:03 jterapin

Current standing:

  • All protocol-tests passes but more work required:
    • We have 58 ignored tests
      • 11 tests cases rely on Smithy fixing the C2J translation (may require additional work after)
      • 4 test cases cannot be fixed due complex issue
      • 43 test cases to be resolved when test runner is updated to handle error cases (need to implement)
  • There are 6 failures in our gem specs after making changes (these need to be investigated and resolve)
  • There are 44 failures in the on build_tools directory (need to investigate and resolve)
  • Clean up code (optimize approaches, etc)
  • Create a comprehensive log of changes (intended for changelog)
  • Create a list of services to manually test certain operations that are affected by some SDK core changes
  • Manual testing/run integration tests (have another team member to run integration tests on their end to verify)
  • Bring over "extra" test cases (aka eventstreams) and update the formatting
  • Update api-gateway protocol-tests formatting if needed
  • CR review and update based on feedback
  • After CR approvals, I'm going to rubocop some of the files I touched since I want to leave the file cleaner than when I found it 🧹 ✨

jterapin avatar Mar 14 '24 21:03 jterapin