Distinction between "runtime" (explainer) metadata and "static" (explanation) metadata

[jklaise]

Both Explainer objects and Explanation objects (returned by calling explain) have a meta attribute storing information about how the Explainer was configured, including information such as user-configured parameters passed to __init__, fit, explain, as well as some extra information about the type of explanation and internal processing. This metadata is intended to be used in two distinct ways:

• As an attribute of Explanation to enable the user to have a complete record of the parameters used to obtain the explanation, as a consequence this meta should be json-serializable and independent of custom Python objects
• As an attribute of Explainer to "keep track" of user parameters, potentially including complex Python objects such as callables or models, which would be serialized and deserialized when an explainer is persisted.

There is a clear difference between the two intended use cases which comes to a head when user-configured inputs are complex Python objects such as callables or models as it highlights that the two meta attributes should really be different, but it is not clear how to map custom objects to json-serializable descriptions when going from Explainer metadata to Explanation metadata. For a concrete example, consider the case when a user passes a custom segmentation function to AnchorImage, we currently simply return a string "custom" in the Explanation metadata: https://github.com/SeldonIO/alibi/blob/242675149b7cc84c38d06d76c2b8557f306ae11f/alibi/explainers/anchor_image.py#L100-L103

We should attempt to have a consistent way of dealing with these two different views of "metadata".

One crucial question to answer is "should it be possible for someone reading the Explanation metadata to completely recreate the original Explainer"? If the answer is "yes" then that would steer the conversation towards mapping complex objects to string descriptors in an invertible manner, e.g. something like https://thinc.ai/docs/usage-config#registry. Note that in this case the user still needs access to the definitions of the complex objects separately from Explanation, but the string descriptors would map unambiguously to the particulal object used (as opposed to the rather cryptic "custom" in the AnchorImage example above). If the answer is "no" then we just need a systematic way of describing "complex" user parameters in a simple string format.