Discrepancy in encoding of JSON between Python and Ruby

[stefankraus-wf]

I am unable to run contract tests between two python services that authenticate using HMAC keys due to the serialization differences between the actual requests and the mocked request provided by the pact-verifier.

I have a Consumer test that signs requests with an HMAC:

request = requests.Request(
  method=’POST’, url=’http://localhost’, json={‘test’: ‘a’})
prep = request.prepare() 
signature = hmac.new(‘abc123’.encode('utf-8'),
                     msg=prep.body,
                     digestmod=hashlib.sha512).hexdigest()
prep.headers['Request-Signature'] = signature
resp = requests.session().send(prep) 
return resp 

And a Provider that verifies that HMAC:

@blueprint.route(‘/’, methods=[‘POST’])
def handle():
  ...
  signature = hmac.new(
    ‘abc123’.encode(‘utf-8’),
    msg=body,
    digestmod=hashlib.sha512).hexdigest()
  # if signature != request.headers.get('Request-Signature`):
    # abort(401)

At the HTTP level, what is being sent from the Consumer is exactly:

‘{“status”: “e”}’

However, what is being received by the Provider is exactly:

‘{“status”:”e”}’

This would not be a problem under normal circumstances since the json encoder is ambivalent to whitespace. However, in the case of HMAC keys where the exact byte sequence of the request body is used it causes failures to authenticate.

The json contract spec that I get from running the Consumer unit tests is:

  "interactions": [
    {
      "description": "<test request>",
      "providerState": "<state>",
      "request": {
        "method": "post",
        "path": "/",
        "headers": {
          "Content-Type": "application/json",
          "Request-Signature": "<sig>"
        },
        "body": {
          "test": "a"
        }
      },
      "response": {
        "status": 200,
        "headers": {
        },
        "body": [
          "Success",
          200
        ]
      }
    },
…
  ]
}

[bethesque]

Your two examples (sent by the consumer, received by the provider) look identical to me, but I assume you mean to show that there is whitespace difference.

Is there a way you can stub out your signing code for the contract tests? Whether or not the signing should be part of the contract is debatable. Could you leave that part of the integration to be tested by actual integration tests?

[stefankraus-wf]

Correct, the whitespace is what I was referring to.

It's entirely possible for me to stub out the signing code when running the tests, though I'd rather keep the service as it is in prod.

My qualm is that the contracts aren't being verified based on what exactly is sent, but rather by how the json is encoded:

json.loads('{"a": "b"}') == json.loads('{"a":"b"}')
'{"a": "b"}') != '{"a":"b"}'

HMAC is just an instance where this discrepancy actually matters.

Note: I worked around this issue by having my service conform to dumping the json content without spaces :: json.dumps(content, separators=(',', ':'))

[bethesque]

I'm glad you've got a work around. It's a tricky situation.

[sergeyklay]

I'm closing this issue due the solution was found but if I have made an oversight, please do get back in touch. Thanks