Convert BigInteger to string in a dictionary for response

[yhosun]

It would be a question or enhancement request.

I was wondering if there is a way to convert Sqlachemy BigInteger to a string instead of a number in a dictionary to include in HTTP response.

In my Sqlachemy model, I have a column with BigInteger - my_id = Column(BigInteger).
For example, if the my_id attribute has a value 13851168950191161, and it will be my_id: 13851168950191161 in a HTTP response. In Javascript, the number becomes 13851168950191160 which is greater than the Number.MAX_SAFE_INTEGER. So, I wanted to convert to a string if the column is BigInteger type, or wanted to find another workaround / solution.

I thought about overriding helpers.to_dict, but it looks like I cannot detect Sqlachemy data type.

Thank you.

[jfinkels]

This is a good question, thanks for sharing!

Flask-Restless has changed quite a bit in version 1.0.0b1. Now you can more easily define a custom serializer class; see the serialization section of the documentation for more information. Specifically, you can override the serialize() method in a subclass of DefaultSerializer, create the response document dictionary by calling the method of the superclass, check for any (Python) integers, and convert them to strings by modifying the response document dictionary and returning it. So that's the answer if this is just a question.

If this is a feature request, presumably the client would receive the correct document, something like

{
  "id": 13851168950191161
}

but the actual Javascript code that loads a Javascript object from that JSON returns an object with id attribute equal to 13851168950191160. Is that the problem you're having?

If so, now the question is whether it makes sense for Flask-Restless to always translate big integers to strings? Or should it be the responsibility of the client?

PS You should be able to get the type of a SQLAlchemy field using the (non-public) helper function get_field_type in the flask_restless/helpers.py module: https://github.com/jfinkels/flask-restless/blob/ec4cd2c0e72b1bae68d83d09505b19fd30a952c5/flask_restless/helpers.py#L226

[yhosun]

Thank you very much @jfinkels !

I will upgrade Flask-Restless to 1.0.0b1 and implement a custom serializer. I would not always want to convert any Python integer to a string, and as you pointed, I will use or implement a similar function as get_field_type to find if it is SqlAchemy BigInteger.

Definitely, it is a client side issue that the a big integer is not handled properly, but it will be very difficult to solve it in a client side unless I send all columns types to Javascript and somehow add double-quotes only for a big integer attribute before parsing the response to create a Javascript object.

It is not a Flask-Restless issue, but it could become a common issue for applications using Javascript in a client-side. I was hoping that Flask-Restless provide a simple (global) way to map a model data type to a type in a dictionary for a response if possible.

Thank you again. It is very helpful.

[jfinkels]

Well, the way it is implemented now is as follows. The DefaultSerializer._dump method creates a Python dictionary representing the response document. Eventually, this dictionary is passed to the flask.jsonify function from Flask, which creates the actual response object (containing the payload as a string). So there are two places where casting to a string could occur, here in the Flask-Restless serializer or there in the Flask JSON encoder. It may be that since this will be a problem for any Javascript client receiving a JSONified response from Python containing a sufficiently large integer, you may want to raise this question at https://github.com/pallets/flask, and see if they have any ideas about where the cast to a string should occur.

As a workaround, you can use the strategy I suggested above. Another workaround might be to create a method or property in your SQLAlchemy model that just returns the integer value as a string, then tell Flask-Restless to include that property and exclude the integer property.

[jfinkels]

In any case, your suggestion is interesting, too:

I was hoping that Flask-Restless provide a simple (global) way to map a model data type to a type in a dictionary for a response if possible.

Let me think about how such a thing would be implemented, and if it even makes sense.