pickle icon indicating copy to clipboard operation
pickle copied to clipboard

Load in JSON

Open AgostinoArcasensa opened this issue 2 years ago • 3 comments

I'm trying to load a wide serie of .sav files using your Nuget, but my attempts ended in this error.

expected zero arguments for construction of ClassDict (for numpy.dtype). This happens when an unsupported/unregistered class is being unpickled that requires construction arguments. Fix it by registering a custom IObjectConstructor for this class

Is it possible to set a default translation in a KeyValuePair<string, string> for the unregistered/unsupported classes, or add a specific constructor for numpy.dtype class?

AgostinoArcasensa avatar Oct 11 '23 16:10 AgostinoArcasensa

Yes. try Unpickler.registerConstructor(...)

However you may find that you need to add a constructor for all numpy types that might be pickled into your ".sav" file. It could be easier to not store the numpy data in your pickles, but convert it to normal python datatypes before writing the pickle file.

irmen avatar Oct 18 '23 16:10 irmen

However you may find that you need to add a constructor for all numpy types that might be pickled into your ".sav" file. It could be easier to not store the numpy data in your pickles, but convert it to normal python datatypes before writing the pickle file.

It might be difficult to edit all our .sav files (not all of them are created by us).

We've defined a JSONConstructor():

public class JSONConstructor : IObjectConstructor
{
    public object construct(object[] args)
    {
        try
        {
            string json = JsonConvert.SerializeObject(args);

            return json;
        }
        catch (Exception x)
        {
            throw new PickleException("problem constructing object", x);
        }
    }
}

But we encounter the following exception:

  Nome Valore Tipo
e {"failed to setstate()"} System.Exception {Razorvine.Pickle.PickleException}

We think the Exception is caused by the missing setstate() method for a primitive object (string).

We even tried to use the Unpickler.registerConstructor with the ClassDictConstructor() method but we encounter the following exception:

  Nome Valore Tipo
e {"expected zero arguments for construction of ClassDict (for numpy.dtype). This happens when an unsupported/unregistered class is being unpickled that requires construction arguments. Fix it by registering a custom IObjectConstructor for this class."} System.Exception {Razorvine.Pickle.PickleException}
e {"expected zero arguments for construction of ClassDict (for numpy.core.multiarray._reconstruct). This happens when an unsupported/unregistered class is being unpickled that requires construction arguments. Fix it by registering a custom IObjectConstructor for this class."} System.Exception {Razorvine.Pickle.PickleException}

Can you please give us some instructions to resolve the Exceptions?

Thank you!

AgostinoArcasensa avatar Oct 21 '23 16:10 AgostinoArcasensa

Try to use the source code to figure out why you still get these errors. For example, here are a couple of unit tests that utilize the custom class constructor registry https://github.com/irmen/pickle/blob/master/java/src/test/java/net/razorvine/pickle/test/UnpicklerComplexTest.java

also you already see for example that you encounter other numpy types such as numpy.core.multiarray that you'll have to deal with.

irmen avatar Oct 21 '23 19:10 irmen