Stubber blocks boto3 clients from raising ParamValidationError

[yukihiko-shinoda]

Describe the bug

Stubber blocks boto3 client from raising ParamValidationError.

Steps to reproduce

None of following test succeed: The step to reproduce issue of botocore Stubber.

Expected behavior

If parameter is not valid, boto3 client raises ParamValidationError even if it's stubbed. Stubber doesn't block parmeter validation process of boto3 client, only the subsequent network process is replaced with stub.

Debug logs

Full stack trace of issue of botocore Stubber.

[stobrien89]

Hi @yukihiko-shinoda,

Sorry to hear about the issues. Just to clarify, in which cases are you expecting a ParamValidationError to be raised? I'm able to follow your code, but it's not clear what's expected based on the test names. Thanks!

[yukihiko-shinoda]

Sorry, test_2() I am: The step to reproduce issue of botocore Stubber.

The boto3 logs client put_log_events() doesn't accept argument sequenceToken=None, then ParamValidationError is raised. Stubber blocks it.

I meant by test_1() and test_3() that just to be sure there are no way to test except block by using Stubber.

[stobrien89]

Hi @yukihiko-shinoda,

Thanks for the clarification. We'll review this as a team next week and provide an update as soon as we have a chance to talk about it.

[kdaily]

Hi @yukihiko-shinoda,

Thanks for your patience. There are a couple of underlying issues.

1. The stubbers assertion of expected parameters happens before the clients own parameter validation to determine if the call is even valid. We might not be able to fix this; the order of parameter assertion can't easily be changed due to how the event system works.

2. Even if you set the expected parameters to the “bad” set that the client validation will reject, the response remains stubbed on client in effect breaking the stubber. It might be possible to address the case where the "bad" expected but not straightforward, since we’ll have to pop the response before we technically know if the underlying call is actually going to raise a ParamValidationError.

The recommendation in this scenario is to use an if statement to avoid this issue when using a ParamValidationError for flow control. Possibly, you may be able to build the kwargs once, and then just always set the events and call with **kwargs.

I'll leave this open as a feature request to see if any others encounter this, or any other changes are made that might alleviate these issues.