botocore icon indicating copy to clipboard operation
botocore copied to clipboard

Support null values in parsers when services accepts and returns null values.

Open adev-code opened this issue 2 months ago • 2 comments

This PR addresses parsing where the _handle_integer method returns with a TypeError when a service (e.g. RDS Data API) supports and returns null or None values.

The root cause lies in the _handle_integer method in RestJSONParser class, which directly calls int(value) without validating whether the value is None or not This creates an inconsistency where successful cases like integers parse correctly, but any value containing null values triggers a TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'.

The fix implements proper null value validation in the integer parsing logic, ensuring that null values are handled gracefully while maintaining backward compatibility with existing functionality. This change allows AWS services to properly support null values.

Fixes: https://github.com/boto/boto3/issues/4508

adev-code avatar Oct 09 '25 17:10 adev-code

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

:x: Patch coverage is 40.00000% with 6 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 93.13%. Comparing base (bbed2c0) to head (4cfb1e7). :warning: Report is 263 commits behind head on develop.

Files with missing lines Patch % Lines
botocore/parsers.py 33.33% 6 Missing :warning:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #3574   +/-   ##
========================================
  Coverage    93.13%   93.13%           
========================================
  Files           68       68           
  Lines        15336    15421   +85     
========================================
+ Hits         14283    14363   +80     
- Misses        1053     1058    +5     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Oct 09 '25 17:10 codecov-commenter

The issue you linked points to the shape being boxed long, which means it is a nullable int in python. This suggests that we have a bug.

I'd hope that we can do a few things to move this forward, but it may be asking too much, so I'm open to some discussion to make sure we can move this forward at a reasonable rate:

  1. We need to add tests. Ideally this comes in the form of upstream additions to the protocol tests for all protocols that support the box trait. That may be all of them, we will need to find a definition of what the box trait does to confirm.
  2. We should make sure this works for all protocols and primitive types, not just integers in RestJson. This likely requires more code changes to ensure all of our _handle* methods can accept a null when it's marked as box.
  3. We have encountered some issues in the past with our parsers being too lenient, we should keep this change impacting as few shapes as possible. Instead of always supporting nulls for shapes modeled as integers/longs, what do you think of changing the logic so that it only accepts null as a valid value if the shape is modeled as box, and keeping the logic the same otherwise?

SamRemis avatar Oct 10 '25 15:10 SamRemis