Add record and replay feature

[szopen321]

proposed solution to https://github.com/spulec/moto/issues/5299

This allows to record calls to moto and replay them to restore the same state.

Feature breakdown:

• endpoints to start and stop recording
• endpoints to reset, download and upload recording
• an endpoint to replay recorded entries
• an endpoint to sed random seed

Notes and limitations:

• not all services are covered by recording. Confirmed covered ones are: ec2, vpc and lambda.
• methods with names starting with "Describe" are ommited in recording as they don't change state
• to recreate randomly generated values (eg. ids of ec2 instances) while replaying one has to set random seed
• recording is stored in a file to which path is stored in an environment variable
• whether recording is by default enabled or not is also stored in an environment variable

[szopen321]

Hi @bblommers! Here is another PR, I hope you'll like it. It seems like using mock.patch doesn't work in server mode. Do you have any solution to that or should I add raise SkipTest to solve this?

[yoctozepto]

Hi @bblommers! I hope you are doing well. Is there any way I could help you with analysing this PR?

[bblommers]

Hi @szopen321 , thanks for the PR! This would be a really useful feature, and the approach makes sense overall.

The next major release will see some big changes, but maybe this can be part of a 4.x release. I'll have another look at the specifics of this PR after 4.0 is released.

As far as the mock.patch goes, it's not possible to mock this in ServerMode. So these should either be skipped, or rewritten as integration-style tests that work in both modes (like in test_set_seed).

[bblommers]

Hi @szopen321, if you don't mind, I'll work on this PR for a bit to see if I can get it to work for other services as well. So don't spend too much time worrying about getting the build to pass.

[bblommers]

@szopen321 What was your usecase for the random seed? The EC2 instance ID's happen to be generated using random.randint, but a lot of ID's are generated using UUID's, which are not affected by the set_seed-method.

I'm a little worried about the confusion this will cause if, when a user calls the set-seed-method, only half of the ID's end up being reproducible, and the other half are still random.

[szopen321]

Hi @bblommers! Thanks for noting that. What would you say about monkey patching UUID.uuid to make it use rand seed like this: uuid.UUID(int=rd.getrandbits(128), version=4) I would add the patching in set_seed method.

[bblommers]

Hmmm.. patching a system-module can introduce a lot of unintended consequences and unexpected behaviour.

It maybe a better option to provide a utility-method that uses a semi-randomized uuid:

def get_uuid4():
    return uuid.UUID(int=rd.getrandbits(128), version=4)

Given the amount of UUID's that are used throughout Moto that is quite a wide-reaching change though. I'll make that change in a separate PR, otherwise it will be impossible to review.

This weekend I'll try and finish a PR that records all requests, for all services.

[bblommers]

Thank you again for getting this started @szopen321! It was incredibly useful to see this feature in action, and to have a good API/interface to guide the final implementation.