request for pluggable json encode/decode functions to process response

[vaswadhati]

Hi there,

Luv bottle so far. Ran into an expected issue when dealing with json encoding that I have fixed for myself temporarily in my local copy.

Couple of issues so far: 1/. It is perfectly alright to json encode a list (or a tuple) for that matter - in the pragmatic world of building Ajaxy web apps. The latest dev version (that I am using) checks if the response is a dict and if so applies json encoding on it. I patched mine to apply json encoding if it is a dict or a list. Not 100% sure about tuples so I left it at that.

2/. The response consists of data retrieved from a database and as expected json encoding tripped when it saw datetime and date types. There is a known way to override json package's dumps and loads methods. I took patch (that I had to write when using Py-rs) and just inserted again into my copy of bottle.

Overall, worked around the issue and I am in good shape to continue to use bottle. What nags me is that I had plans of extending my json patch to support direct encoding of database results from the database layer API such as straight dbapi, sqlalchemy - something along the lines of what I found at http://www.mail-archive.com/[email protected]/msg03926.html - which means I will have to keep patching bottle as new versions get released.

Ideal would be if I could pass json encode and decode functions when Bottle() instantiated - something along the lines of:

myapi = Bottle(json_encode = myencoder, json_decode=mydecoder);

Or some thing along those lines or better. You get the idea.

Making an automatic decision to encode to json based on response type is a pretty significant and important design decision. And I think should come with an option to allow the user to configure what gets encoded and how.

Thanks,

[iurisilvio]

The autojson feature is handled with JsonPlugin. You have some options to handle your issue, avoiding changes to Bottle core:

1. Just override `bottle.JSONPlugin with any plugin you want;

2. Start Bottle(autojson=False) and install json plugin with your custom dumps function JSONPlugin(json_dumps=mydumps). It is configurable per app, better than first one.

3. Start with autojson=False and install your own custom JSONPlugin. This way you can enforce list dump and anything else you want, with full control of bottle response.

This plugin is really simple, you can take a look at it in [1] to just create your own.

Of course another solution is add your keywords to json dump in Bottle instantiation.

[1] https://github.com/defnull/bottle/blob/master/bottle.py#L1501

[defnull]

Or simply::

for plugin in app.plugins:
    if isinstance(plugin, bottle.JSONPlugin):
        plugin.json_dumps = my_custom_json_dumps

But you are right, this should be more obvious. I'll keep this issue open.

[vaswadhati]

Thanks for the consideration. Fyi, py-rs [1](from which I moved away from for other reasons) supports encoding and decoding like so:

@rs.post
@rs.consumes('application/json', json.loads)
def foo(entity=rs.context.entity): pass
# foo function will accept "POST /" request only if its Content-Type
# header matches consume media value  'application/json'. The value
# of entity parameter will be set to json.loads(request.entity) result.

@rs.get
@rs.produces('application/json', json.dumps)
def foo():
  return {'id':1, 'name':'Entity', 'zip':None}
# foo function will respond to "GET /" request with 
# json.dumps({'id':1, 'name':'Entity', 'zip':None}) result as HTTP Response Entity.

In the above scheme I could easily replace json.dumps with my_encoder and json.loads with my_decoder. Not suggesting that bottle do the same, but something along those lines or better will be good so that users don't have to patch bottle with each release. One thing I found wanting in the above scheme is being able to pass in optional arguments to the encoding or decoding function - which currently py-rs doesn't support.

[1] http://code.google.com/p/py-rs/wiki/Concept

[fallenpegasus]

Could someone post an example of using JSONPlugin with the MongoDB pymongo bson json_util ?

[dzerrenner]

As response to fallenpegasus's question (i know, a bit late, but i just had the same problem and found this request): You can register the json_util.default like this:

from bottle import Bottle, JSONPlugin
from bson import json_util
from json import dumps

app = Bottle(autojson=False)
app.install(JSONPlugin(json_dumps=lambda s: dumps(s, default=json_util.default)))

Now you are able to return a document retreived from mongodb without further modification.

[HarryR]

Even later here :)

This ticket can probably be closed as there are a couple of good solutions.

import json, datetime, bottle
class MyJsonEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return str(obj.strftime("%Y-%m-%d %H:%M:%S"))
        return json.JSONEncoder.default(self, obj)

app.install(bottle.JSONPlugin(json_dumps=lambda s: json.dumps(s, cls=MyJsonEncoder)))

[r-pluss]

I think that I'm missing something here: I generally provide simplejson with a lambda expression for the default argument to handle things like datetime.datetime and datetime.date . This works well in other places. However, trying to use it with the above examples is not working for me, I still get errors akin to datetime.datetime(x,x,x,x,x) is not JSON serializable.

dt_handler = lambda obj:(obj.strftime('%Y-%m-%d %H:%M:%S') if isinstance(obj,datetime.datetime) else (obj.strftime('%Y-%m-%d') if isinstance(obj, datetime.date) else None))
app=Bottle(autojson=False)
app.install(JSONPlugin(json_dumps = lambda s: dumps(s, default= dt_handler)))

Any guidance?

[eevleevs]

Is there any way to override also json_loads?