Gracefully reraise JSONDecodeError errors

[askedrelic]

We've run into this issue several times, where a swagger service has crashed and is returning HTML 500s, which then causes our client to raise JSONDecodeError.

These are pretty ugly and hard to debug; you can't see what the not-json response is.

I feel like the bravado client should handle this case and reraise as a Bravado custom Exception, with the crash context/original response available.

Thoughts? I will leave this open, but don't have immediate plans to work on this.

Here is a simplified crash:

File "virtualenv_run/local/lib/python2.7/site-packages/swaggerpy/client.py", line 159, in get_client
    return cache(*args, **kwargs)
File "virtualenv_run/local/lib/python2.7/site-packages/swaggerpy/client.py", line 118, in __call__
    self.build_client(*args, **kwargs), ttl)
File "virtualenv_run/local/lib/python2.7/site-packages/swaggerpy/client.py", line 124, in build_client
    return SwaggerClient.from_url(api_docs, *args, **kwargs)
File "virtualenv_run/local/lib/python2.7/site-packages/swaggerpy/client.py", line 326, in from_url
    load_resource_listing(url, http_client, None, request_options),
File "virtualenv_run/local/lib/python2.7/site-packages/swaggerpy/swagger_model.py", line 196, in load_resource_listing
    add_api_docs(resource_listing)
File "virtualenv_run/local/lib/python2.7/site-packages/swaggerpy/swagger_model.py", line 184, in add_api_docs
    api['api_declaration'] = eventual.wait(timeout=timeout).json()
File "virtualenv_run/local/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
File "virtualenv_run/local/lib/python2.7/site-packages/simplejson/__init__.py", line 505, in loads
    return _default_decoder.decode(s)
File "virtualenv_run/local/lib/python2.7/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
File "virtualenv_run/local/lib/python2.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[laucia]

This was probably fixed by https://github.com/Yelp/bravado-core/pull/97

[sjaensch]

@laucia: Yelp/bravado-core#97 is about fixing JSONDecodeErrors when reading JSON data submitted by the client. This is about trying to fetch swagger specs, the server returning a 500 (with a HTML response) and then bravado trying to parse that HTML as JSON. We still have to fix that.

[bxm156]

I have encountered this as well. One of the APIs I hit allows me to pass a field called "debug", and instead of returning a JSON response, it can return some text data to aid with debugging. I would like to be able to read this text data for my own investigations, but I can't because Bravado would just raise JSONDecodeError.

[helgridly]

I dug into this a little bit and managed to fix it locally, for my purposes, in #249.

[macisamuele]

If your endpoint should produce JSON output (produces in SwaggerObject or in OperationObject) and instead it produces HTML output the trouble is on the swagger specs and not on the bravado client. If it is the case the specs should be updated ... once you do this you'll face trouble because currently bravado-core is handling only JSON responses (similar issue #250).

The idea could be to have a more relaxed bravado-core which is not trying to execute the unmarshal_response if the system doesn't know how to handle a specific content.

[bxm156]

What about a case where there is a load balancer between you and your JSON only endpoint. If the load balancer returns a status of 500, and a 500 HTML page, you will never know its a 500. Even when using "also_return_response': True" to get the http status, bravado will raise a JSONDecodeError, preventing you from even seeing a status code.(See https://github.com/Yelp/bravado/issues/250)

Could we gracefully handle this so we can get at minimum HTTP status code of the response, even if a JSONDecodeError is raised.