How to test failures/exceptions

[ghostsquad]

How might one use this framework to also test exceptions being raised? For instance, trying to terminate an instance that is already terminated.

[ghostsquad]

I think I answered my own question here. This is framework is pretty slick. :+1: Because only the http response is what is being "replaced", boto properly handles saved responses when errors occur. Maybe some extra documentation around this would make the documentation complete and answer this question for other people looking to use this framework.

Also, something else to be aware of. I'd recommend a note to rename "~/.aws/credentials" to something else for the duration of the test, just as a paranoid precaution.

Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
>>> import placebo
>>> session = boto3.Session()
>>> pill = placebo.attach(session, data_path='/tmp/tests/resources/aws/responses')
>>> pill.playback()
>>> asg = session.client('autoscaling')
>>> asg.terminate_instance_in_auto_scaling_group(InstanceId='i-02b76bda', ShouldDecrementDesiredCapacity=True)
{'ResponseMetadata': {'RequestId': '7a8115c2-d691-11e5-b273-57f1c85f6e25', 'HTTPStatusCode': 200}, 'Activity': {'Description': 'Terminating EC2 instance: i-02b76bda', 'StatusCode': 'InProgress', 'StartTime': datetime.datetime(2016, 2, 18, 22, 46, 41, 879000), 'Details': '{"Availability Zone":"us-west-2a","Subnet ID":"subnet-4e883339"}', 'Cause': 'At 2016-02-18T22:46:41Z instance i-02b76bda was taken out of service in response to a user request, shrinking the capacity from 1 to 0.', 'ActivityId': '4a31f31e-979a-4fb7-881d-29a3323a4359', 'Progress': 0}}
>>> asg.terminate_instance_in_auto_scaling_group(InstanceId='i-02b76bda', ShouldDecrementDesiredCapacity=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/botocore/client.py", line 310, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/botocore/client.py", line 407, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the TerminateInstanceInAutoScalingGroup operation: Instance Id not found - No managed instance found for instance ID i-02b76bda

[garnaat]

Yeah, recording failed requests should work just fine and replaying those should create the same error condition. I agree more docs would help (always!).

Perhaps when in playback mode placebo could monkey around with things to make sure it is not finding any credentials anywhere?

[ghostsquad]

That would be nice. In my tests (using pytest) I'm doing this:

user_home_dir = os.path.expanduser('~')
credentials_file_path = os.path.join(user_home_dir, '.aws/credentials')
new_credentials_file_path = credentials_file_path + '.bak'

@pytest.fixture(scope="module")
def rename_credentials_file(request):
    try:
        if os.path.exists(credentials_file_path):
            os.rename(credentials_file_path, new_credentials_file_path)
    except Exception:
        print("WARN: unable to rename aws credentials file: {}.".format(credentials_file_path))

    def fin():
        if os.path.exists(new_credentials_file_path):
            os.rename(new_credentials_file_path, credentials_file_path)

    request.addfinalizer(fin)

if you think something like this would be helpful, I'd be happy to submit a pull request.

[garnaat]

I'd rather not actually move the file around, if possible. Perhaps we could use the corresponding environment variables (e.g. AWS_CONFIG_FILE or AWS_SHARED_CREDENTIALS) to force it to ignore your normal credentials.

[jantman]

I just happened to stumble on this project - very cool.

I can suggest a few ways of ensuring that boto3/botocore won't read credentials from ~/.aws:

1. If you export the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, botocore.credentials.create_credential_resolver() will never move on to loading from files.
2. Since placebo has access to the Session object, it could simply set the credentials_file and config_file session config variables to non-existent paths, preventing them from being loaded.
3. In a more generic method, you can simply set os.environ['HOME'] to an empty skeleton directory before running tests, and ~/ will expand to that path.

[ghostsquad]

Hmmm I like this idea. I admit, my solution is a bit of a hack.