XML instead of JSON reply

[janelogan]

Hi,

Thanks for developing CMB and making it open source.

I just started playing with it and discovered that it is returning XML instead of JSON to my SNS requests (sent via the python aws library boto).

For example, BOTO sent out something like:

GET /?Action=GetTopicAttributes&ContentType=JSON&TopicArn=arn%3Acmb%3Acns%3Accp%3A392881562674%3Atest&Version=2010-03-31 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nHost: sns.us-east-1.amazonaws.com\r\nAuthorization: AWS4-HMAC-SHA256 Credential=NLHE68VKGI66AS6GNSMN/20140224/us-east-1/sns/aws4_request,SignedHeaders=host;x-amz-date,Signature=29e34b31276772feda3d65efb48ccc7a4f8db2033fa5fab2b48d00b73dce6f7e\r\nX-Amz-Date: 20140224T103234Z\r\nUser-Agent: Boto/2.25.0 Python/2.6.6 Linux/2.6.32-26-pve\r\n\r\n

CMB replied with some XML, like:

Content-Type: text/xml
Content-Length: 1745
Connection: keep-alive
Server: Jetty(8.1.10.v20130312)

<GetTopicAttributesResponse>
<GetTopicAttributesResult>
<Attributes>
<entry>
<key>EffectiveDeliveryPolicy</key>
etc.

AWS replied with some JSON, like:

x-amzn-RequestId: 4b90e267-710b-5eda-a2ec-a9e282776605
Content-Type: application/json
Content-Length: 1127

{"GetTopicAttributesResponse":{"GetTopicAttributesResult":{"Attributes":{"DisplayName":"","EffectiveDeliveryPolicy":"
etc.

As a result, boto worked ok with AWS but returned this error with CMB:

ERROR No JSON object could be decoded

I am already using the latest CMB on github with the boto fix (06baa74a32). Any suggestion or insight into this problem much appreciated.

Kind regards, Jane

[boriwo]

The boto fix was a minor fix for cqs and you can currently use cqs with boto. We are aware of the remaining issues with boto and cns: Basically boto sns uses the undocumented ContentType=JSON parameter to receive JSON encoded responses from AWS and CMB currently does not support that. The same question has come up before and we may add JSON support to CNS in the next release to be compatible.

[janelogan]

Thanks for replying so quickly. It will be nice if CNS can work with boto in the future :)

For others who are also having this ContentType=JSON problem, there is a patch for boto https://github.com/dmcgowan/boto/commit/659718d4834b529147fe073b48baa868a9d10763 that supposedly work with CNS.

Unfortunately, it is based on a 2 year old version of boto (2.5.1) The author mentioned about 2 months ago that he might work on merging it into boto https://github.com/boto/boto/pull/827 ... so fingers crossed ;)

[dmcgowan]

Boris and I have actually been chatting about this. I could merge in the upstream and re-attempt the pull request but there is no guarantee that the pull request will be merged (it wasn't 2 years ago when there was no conflicts). At least with adding the ContentType we have control of that change while there are no guarantees with boto. My fork should work with CNS, and if there are issues I will fix those in the fork.

[janelogan]

Hi Derek, thanks for pitching in.

I did try replacing boto 2.25.0 with your boto fork, but I have not managed to test the CNS feature since the python app I have been given seemed to require some functions in newer boto releases.

For example, it made some S3 calls using your boto fork before CNS calls and could not find functions like boto.s3.regions():

if not self.aws_region in (r.name for r in boto.s3.regions()):
AttributeError: 'module' object has no attribute 'regions'

As you mentioned quite rightly, it might be easier to add ContentType=JSON to CMB than patching boto.

Kind regards, Jane.