Replace integration tests with unit tests

[nozzlegear]

After years of woe attempting to run integration tests for this package, I'm announcing my intention to completely remove the integration tests and just replace them with plain old unit tests. As it stands, the integration tests are ridiculously flaky and fickle, often failing for random reasons or timing out because of arbitrary limits Shopify puts on the development store the tests run on.

Every time I merge a pull request or push commits to this repo, I need to restart the tests 2-3 times because they'll randomly fail on different endpoints. Lately I've been getting a ton of "connection with the server was terminated abnormally" exceptions that will disappear on the next rebuild. Each build/rebuild takes 10-15 minutes between runs, which ends up being a considerable time investment for me. Not to mention the countless emails I'm getting from my test store about new orders, every time the test suite runs. 😅

It's my opinion that the integration tests haven't really been very useful, either. I can't recall any time they've caught an actual bug versus just showcasing the limits of development stores. It's impossible (or at least incredibly tedious) for contributors other than myself to run the tests, so often we'll receive pull requests that contain tests which have never been run. That moves the burden of writing tests that pass to myself -- which I don't mind, but I often don't have the time to do it.

So with all of that in mind, I'm going to slowly replace the integration tests (the tests that call the Shopify API) with unit tests. The new tests will be responsible for confirming that all data objects, URLs and parsing done by this package match what is documented by Shopify; they just won't send that data to Shopify. We'll trust that the API behaves the way Shopify says it does.

For examples of the kinds of unit tests I'll be writing, take a look at Stripe.Net's test suite and the unit tests that I've written for the ShopifySharp.Experimental package (although they'll be in C# instead of F#).

[clement911]

Integration tests are a good idea in theory but I've also encountered a lot of brittleness whenever I tried them, so I totally understand your reasoning.