[gk] - adding endpoint invocation tracking and querying admin resource

[gregkowalski]

Hi Eric,

First off, just want to thank you for creating stub.by - it's a great tool and it has been proving itself very useful at our company. We have many teams working on applications that integrate with other teams' apps using RESTful services so you can imagine how much value your tool has added, especially in our dev and testing automation environments.

There is a feature that we would find useful, which I wasn't able to find in the existing stubby implementation. This feature is the ability to validate data that gets sent to stubby, and the ability to track (and query) all invocations that were performed against stubby.

This pull request implements that feature, and we basically keep a track of an Invocation object every time a stub is invoked. You can have a look at the Invocation class below but it's very similar to the Endpoint class. These invocations are then kept in memory, in a dictionary of dictionaries of lists (just for the purposes of indexing and performance) and can be queried through the admin interface at the /invocations/ path.

We have an internal NuGet server where I will share this code with the rest of our organization but I thought you may be interested in having a look at this as well. Let me know what you think, if you'd be interested in including these changes in your project, and if so, if there's more work you'd expect before you accept the pull request and add this feature to NuGet.

Either way, thank you once again, you're making lives easier, at least at our company :)

Greg

[mrak]

This looks like a convenient feature. Thanks for writing tests, too! I see that you want to be able to query the request/response invocations that were received by stubby as a logging feature.

Could you describe how you use this feature? Do you want to query the admin interface for invocations to make sure it's receiving what you think it is? Would it be more convenient to log to yaml files (like the config file's format) for easier greping and tail -fing? We have a --debug flag in one of our other stubby projects that was requested, but this functions on STDOUT instead of to files or in-memory.

I like the idea of increased visibility to how stubby is operating.

A few things to clean up before a merge:

• indent with spaces
• clean up unneeded using lines
• README.md section describing the feature

[gregkowalski]

Here's a small diagram describing our use-case for this feature:

The idea is that we want to execute a functional test for the Business App. We use stubby to return some data from the GET and make the POST work but we also want to verify that the POST contained the correct data. These POST's can be higher volume so we tend to batch them, resulting in the same stubby URL being POST'ed to multiple times with different data.

When the operation finishes, we query the /invocations/resource1 and /invocations/resource2 to retrieve the details of those calls to stubby so that we can verify they contained the expected data in each batched call.

One thing I don't particularly like about my solution is that these invocations are kept in-memory, and are never cleaned up. I suppose it's not a major concern for us as we start-up and tear-down an instance of stubby for each test suite so it's never a long-lived process. Perhaps, this could be more detrimental for other users.

I like the idea of logging to yaml files instead of (or in addition to in-memory) because as you said, we then wouldn't need to go through another admin web interface to get it and could just parse the files (grep, tail, etc.) directly. It probably wouldn't be very difficult to make both the in-memory and yaml files logging turned on based on a config flag (or command line arg).

I'll try to work on this in the next few days, as well as indenting with spaces, cleaning up the using lines, and adding the README.md section with a description.

[mrak]

Thanks for the diagram. Very informative.

Do you use the HTTP interface for the admin portal or the programmatic calls? Do you do assertions on specific properties of the invocation or just that it occured x times?

[gregkowalski]

At the moment, we use the HTTP interface for the admin portal but since we are hosting stubby within the testing assembly, we could switch to the programmatic calls very easily (and it probably would make things more efficient).

We do assertions on properties of the invocation, and more specifically we verify that each invocation contained the correct data in the payload.

[sledoux]

Just created an issue #12 without seeing there was a pullrequest already for that. I really need that feature for our tests as well. Any updates?

[mrak]

As it stands now the changes are quite difficult to read at a glance, as all of the formatting and whitespace was changed. Matching these with the original project will make review much easier.

A README change that describes the use of this feature in the same style as the existing documentation would also help.

The feature is a good one, but this PR is not easy to review or audit, so it dropped off of my radar as my life outside of open source has gotten busier.