httpbin test failures

[bergmark]

Moving https://github.com/commercialhaskell/stackage/issues/5470 here as we've disabled the test suite on stackage. Original:

Test suite failure for package http-client-0.6.4.1
    spec:  exited with: ExitFailure 1
Full log available at /var/stackage/work/unpack-dir/.stack-work/logs/http-client-0.6.4.1-test.log



    Network.HTTP.Client
      Client
        works
        method in URL
          success
          failure
        redirects
          follows redirects FAILED [1]
          allows to disable redirect following FAILED [2]
        managerModifyResponse
          allows to modify the response status code
          modifies the response body
        managerModifyRequest
          port
          checkResponse
          redirectCount FAILED [3]

    Failures:

      test/Network/HTTP/ClientSpec.hs:42:13:
      1) Network.HTTP.Client.Client.redirects follows redirects
           expected: Status {statusCode = 200, statusMessage = "OK"}
            but got: Status {statusCode = 404, statusMessage = "Not Found"}

      To rerun use: --match "/Network.HTTP.Client/Client/redirects/follows redirects/"

      test/Network/HTTP/ClientSpec.hs:49:13:
      2) Network.HTTP.Client.Client.redirects allows to disable redirect following
           expected: Status {statusCode = 302, statusMessage = "Found"}
            but got: Status {statusCode = 404, statusMessage = "Not Found"}

      To rerun use: --match "/Network.HTTP.Client/Client/redirects/allows to disable redirect following/"

      test/Network/HTTP/ClientSpec.hs:96:13:
      3) Network.HTTP.Client.Client.managerModifyRequest redirectCount
           expected: Status {statusCode = 302, statusMessage = "Found"}
            but got: Status {statusCode = 404, statusMessage = "Not Found"}

      To rerun use: --match "/Network.HTTP.Client/Client/managerModifyRequest/redirectCount/"

    Randomized with seed 362566986

    Finished in 0.2514 seconds
    10 examples, 3 failures    

[robx]

It seems like we should be running an http server ourselves in the tests instead of relying on an external service. (Just ran into this myself trying to test https://github.com/snoyberg/http-client/pull/477.)

Would it be acceptable to add say warp and wai as dependencies to the test:http-client:spec target?

[snoyberg]

I disagree, testing against an external service makes sure we haven't hard-coded dependencies on how something like warp behaves.