Upgrade to marshmallow 3

[sloria]

👋 Just dropping by to let you know that marshmallow v3 is released.

Upgrading will provide a few benefits for boundary-layer:

• No need for StrictSchema. v3 has first-class support for unknown field validation, and the default behavior matches StrictSchema: https://marshmallow.readthedocs.io/en/stable/quickstart.html#handling-unknown-fields
• List(Nested(...)) has been improved to have the same validation and performance characteristics asNested(many=True) (List(Nested) was previously slower due to Nested(many=True) taking an optimized path)

https://github.com/etsy/boundary-layer/blob/9b0995625e12a9401788b5f22d78df28e94221ea/boundary_layer/schemas/dag.py#L87-L89

• You can get rid of places where load_from and dump_to match, since they've been merged into a single parameter, data_key.

https://github.com/etsy/boundary-layer/blob/5e52e1cbf89af97ea6d93c86fe081bff3ba89bfe/boundary_layer_default_plugin/oozie_actions.py#L86-L87

• Serialization performance is significantly improved compared to v2

I've only skimmed the boundary-layer code, but it looks like the migration should be relatively straightforward.

1. Use data_key instead instead of load_from and dump_to.
2. Change usages of .load and .dump to handle ValidationError and expect the (de)serialized data dictionary to be returned.

        try:
            data = OozieWorkflowSchema(context={
                'cluster_config': cluster_config,
                'oozie_plugin': oozie_config,
                'macro_translator': JspMacroTranslator(oozie_config.jsp_macros()),
                'production': self.production,
            }).load(parsed)
        except ma.ValidationError:
            raise Exception('Errors parsing file {}: {}'.format(
                filename,
                loaded.errors))

        data_copy = data.copy()

3. Add **kwargs to decorated methods.

    @validates_schema
    def validate_template_undefined(self, data, **kwargs):
        # ...

    @post_dump
    def dagrun_timeout_to_timedelta(self, data, **kwargs):
        # ...

A full upgrading guide is here: https://marshmallow.readthedocs.io/en/latest/upgrading.html

It's worth knowing that marshmallow 3 only supports Python 3. So you'd need to drop support for 2.7 in order to upgrade. Join the party https://python3statement.org/ ! 😄

[dossett]

Thanks @sloria, especially for the overview and migration tips! We hop to be taking the py3 plunge sometime soon. Cheers!

[aksswami]

Please check #57 , once merged it should resolve this issue.

[2timesjay]

Hah, this just came up for us because Langchain requires Marshmallow > 3.3