Is it possible to perform `json.dumps(obj_proxy)`?

[Trung0246]

From reading https://github.com/GrahamDumpleton/wrapt/issues/245, looks like cpython internally doing type check, not class check. The thing is I can't really touch json.dumps call site for my use case (and maybe json.JsonEncoder.default). Unsure how to go about this.

[GrahamDumpleton]

Can you provide a small standalone code example to demonstrate the problem you are having. That will give me something to work with in fully understanding the issue and see what solutions there may be.

[Trung0246]

import wrapt
import copy
import json

class Wrapper(wrapt.ObjectProxy):
	_metadata = None
	def __init__(self, wrapped, data):
		super().__init__(wrapped)
		self._metadata = data

	def __deepcopy__(self, memo):
		return Wrapper(copy.deepcopy(self.__wrapped__, memo), copy.deepcopy(self._metadata, memo))

print(json.dumps({"a": {"a": Wrapper({"b": "123453"}, {"tag": "asd", "id": "555"})}}))

This is my current code so far. Don't really know how to tackle this. I have reduced from my original codebase to this minimal example.

[GrahamDumpleton]

Is __deepcopy__ part of your attempt to get it working, or a required part of what you need independent of needing to convert it to json.

[Trung0246]

It was another independent part but I left it there to hopefully make it works but not I guess. Removing it will cause same error anyways.

[GrahamDumpleton]

Is the goal for the resulting JSON to only show the representation of the object wrapped by ObjectProxy instance, or are you expecting the metadata from the custom ObjectProxy instance to also show up in the JSON output.

[Trung0246]

I don't expect _metadata to show up in the final json string. The only output should be is {"a": {"a": {"b": "123453"}}}, but hopefully can generalize to also include _metadata into the json string.

And hopefully it can works with recursive Wrapper like json.dumps({"a": {"a": Wrapper({"b": Wrapper("123453")})}}) as long as it's the type json.dumps supported.

[GrahamDumpleton]

In what I think is quite hilarious behaviour, if you add indent=4 option to json.dumps() it works.

IOW, running:

import wrapt
import copy
import json

class Wrapper(wrapt.ObjectProxy):
        _metadata = None
        def __init__(self, wrapped, data):
                super().__init__(wrapped)
                self._metadata = data

        def __deepcopy__(self, memo):
                return Wrapper(copy.deepcopy(self.__wrapped__, memo), copy.deepcopy(self._metadata, memo))

print(json.dumps({"a": {"a": Wrapper({"b": "123453"}, {"tag": "asd", "id": "555"})}}, indent=4))

yields:

{
    "a": {
        "a": {
            "b": "123453"
        }
    }
}

In general though, one has to use a custom encoder for json which needs to be supplied when you call json.dumps(). You can write the custom encoder to look for a special method you add to wrapper types if desired so keep how to encode it local to wrapper code. The name of the special method doesn't matter, I chose to use __to_json__().

import wrapt
import copy
import json

class Wrapper1(wrapt.ObjectProxy):
    _metadata = None

    def __init__(self, wrapped, data):
        super().__init__(wrapped)
        self._metadata = data

class Wrapper2(wrapt.ObjectProxy):
    _metadata = None

    def __init__(self, wrapped, data):
        super().__init__(wrapped)
        self._metadata = data

    def __to_json__(self):
        d = copy.copy(self.__wrapped__)
        d["_metadata"] = self._metadata
        return d

# Extend the custom encoder to handle ObjectProxy

def custom_encoder(obj):
    if isinstance(obj, wrapt.ObjectProxy):
        to_json = getattr(obj, "__to_json__", None)
        if to_json: return to_json()
        return obj.__wrapped__
    raise TypeError(f"Object of type {obj.__class__.__name__} is not JSON serializable")

data = json.dumps({
  "a": {"a": Wrapper1({"b": "123453"}, {"tag": "asd", "id": "555"})},
  "b": {"a": Wrapper2({"b": "123453"}, {"tag": "asd", "id": "555"})}
}, default=custom_encoder)

print(data)

This yields:

{"a": {"a": {"b": "123453"}}, "b": {"a": {"b": "123453", "_metadata": {"tag": "asd", "id": "555"}}}}

but which doesn't work as intended again if use indent option. 🤦

[Trung0246]

Thanks. Looks like the only option I have is override json.JSONEncoder.default since I can't touch the call site of json.dumps anyways. Will leave this open when "proper" behavior or implemented in wrapt itself.

[GrahamDumpleton]

Not sure there is much wrapt can do.

The problem is that when you don't supply indent, the json module uses a C implementation of the encoder and that doesn't use isinstance() checks like the pure Python version, and appears instead to do exact type checks (likely for performance reasons). So your wrapper object will pass isinstance(obj, dict) test okay when pure Python version is triggered when indent is supplied and thus why it outputs okay in that case.

So the difference in behaviour is the fault of the Python standard library in that the C version of the json encoder is not friendly to Python duck typing.

BTW, how buried is the point where json.dumps() is called. In extreme case could do monkey patching where that call is made. 😰