how to deal with snake case vs. camel case in Python vs. JSON?

[twosigmajab]

Hi Rebar maintainers and community, just wondering, how are y'all coping with marshmallow-code/marshmallow#1295?

Are we all just reinventing this wheel slightly differently in our own code? Settling for snake_case fields in our JSON or (PEP-8-violating) camelCase fields in our Python? Something else? Thanks in advance for any helpful discussion 😅

[Sytten]

Usually what I have seen in the wild is that python based backends return snake case JSON and the frontend deals with it. But it's most likely easy to create a converter between the two on (de)serialization.

[barakalon]

+1 to @Sytten 's response.

For what its worth, the PEP-8 rule only applies to names, not strings. So {"myAttribute": foo.my_attribute} is compliant.

And If seeing any camelCase at all in your code makes you squeamish, maybe this excerpt from The Zen of Python will help you find solace:

practicality beats purity.

Also, an opinion (although I believe many will agree): It's generally a good practice to maintain a boundary between your business logic layer and your network API layer. Explicitly converting between business logic objects in Python (that might have snake_case attributes) to JSON objects for API response payloads (that might have camelCase attributes) is a good idea. Despite requiring a bit more boilerplate code, this has many benefits, including allowing the business logic to evolve separately from the API schema.

And, similarly, it's often wise for clients to explicitly load JSON objects returned from a server into their own objects, which might include explicitly converting snake_case attributes to camelCase.

[RookieRick]

I don't really have anything to add :) Just assigning the "question" label in keeping with my "all issues shall get labels" policy 😂 Will keep this one open for as long as people wanna discuss. Cheers!

[twosigmajab]

Thanks for thinking about this! Would you accept a PR that added a mixin like the one in marshmallow-code/marshmallow#1295 (similar to the other Schema helpers that Rebar provides in validation.py) to avoid each Rebar user having to figure out a boilerplate solution independently (or come up with their own worse solutions)? /cc @DLuna in case you want to chime in!

[Sytten]

What I would suggest is to allow the customization of Schema helpers (if that is not already the case, I don't remember) and write your validator as a plugin (similar to what I did for auth0). I don't think it should go in the core since this is very dependent on the team.

[twosigmajab]

Steven Loria just added a CamelCaseSchema recipe to the docs, so at least the recipe I posted in marshmallow-code/marshmallow#1295 will now be somewhere in the official Marshmallow docs. It still seems less than ideal for every user facing this to have to copy/paste this into their own code, but if it doesn't belong with the other Schema subclasses in Rebar (e.g. RequestSchema, ResponseSchema), I'm not sure where else it would belong.

[RookieRick]

Just to briefly chime in; I'm not 100% convinced either way myself on the "should/n't we include a helper for this", but if we can figure out a good location to include something that:

1. Doesn't introduce any dependencies
2. Doesn't change default behavior
3. Is general enough to be broadly applicable then I wouldn't see why we couldn't/shouldn't build in support for this. As one of the key "selling points" I see for Rebar is that it makes it easier for developers to construct nice, documented, consistent APIs, something that makes it easier for an API to expose stuff in a particular format (due to the expected consumers) I could see fitting into that mission. All that said, I'm in an "all hands on deck" project that's going to eat 100+% of my time for the next week and a half, so it'll be a bit before I can really dive back in on this. So, consider this my promise that I'm setting a calendar reminder at this point 😂

[twosigmajab]

Thanks @RookieRick, and good luck with what you're working on!

[RookieRick]

So I spent some time playing with this today.. At one point I thought I had it figured out but then it also looks like that approach would break as soon as we build in support for Marshmallow 3.0.. Little did I realize at first that for Marshmallow 2.0 I was kind of exploiting the fact that it seems they call on_bind_field more frequently than needed in 2.0 and fixed that in 3.0.. So I'm now back to leaning toward the "at least we could add something to Recipes" (and/or maybe a helper Mixin in the flask_rebar.utils namespace) approach unless some other burst of inspiration hits me over the weekend :)

[RookieRick]

So to expand on this a bit.. A synopsis of what I had tried on Friday can be seen in this commit: https://github.com/plangrid/flask-rebar/pull/126/commits/0eb94aacb62931c8ef312ac61810effdc7be00e1 - it seemed like a promising approach at first, but as noted in the previous comment, Marshmallow 3.0 gets smarter about how often it needs to actually look at things like on_bind_field which ends up introducing a chicken-and-egg problem when trying to bolt on "schema helpers" after the schema has already been instantiated.

At this point, the best idea that is still rattling around in the back of my mind is something like a CamelCaseMixin designed as an augmentation for marshmallow.Schema (which could incorporate the different logic needed for marshmallow v2 vs v3, for example something similar to the following (which was the "schema_helper" I tried to use in my testing):

def _on_bind_field(schema, field_name, field_obj):
    if LooseVersion(marshmallow_version) < LooseVersion('3.0.0'):
        field_obj.load_from = camelcase(field_name)
        field_obj.dump_to = camelcase(field_name)
    else:
        field_obj.data_key = camelcase(field_obj.data_key or field_name)

Since it's entirely marshmallow-related, I'm also leaning toward Sytten's suggestion that this could/should be a separate optional library that users of Flask-Rebar (or any other marshmallow thing) could incorporate when they define their schema subclasses. What do you think @twosigmajab , you want to add that to your OSS resume and we could link to it from Rebar's docs? 😀

[airstandley]

Update: We've starting using something similar to this internally, see the link for the v3 example. I'm not opposed to adding it as a util, but I like @RookieRick suggestion that it probably beings in a separate library of general Marshmallow utils.

def camelcase(s):
    parts = iter(s.split("_"))
    return next(parts) + "".join(i.title() for i in parts)


class CamelCaseMixin:
    """
    Converts camelCase input to snake_case. And converts snake_case output to camelCase.
    """

    def on_bind_field(self, field_name, field_obj):
        """
        Snake case field names on input and camel case it on output
        Modified from version 3 ex:
        https://marshmallow.readthedocs.io/en/latest/examples.html#inflection-camel-casing-keys
        :param field_name:
        :param field_obj:
        :return:
        """
        field_obj.dump_to = camelcase(field_name)
        field_obj.load_from = camelcase(field_name)