Algorithms cannot be pickle'd

[fschlimb]

Describe the bug Exception is raised when pickling algorithm objects.

To Reproduce

import daal4py as d4p
import pickle
a=d4p.qr()
p=pickle.dumps(a)

Expected behavior It should create a pickle buffer

Output/Screenshots

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "stringsource", line 2, in _daal4py.qr.__reduce_cython__
TypeError: no default __reduce__ due to non-trivial __cinit__

Environment:

• OS: Linux
• Version: 2020.3

[alejandrods]

Is there any update on this? I am facing the same error with:

Python 3.8 daal4py 2021.4.0

[Alexsandruss]

There is no obvious reason to pickle objects like d4p.{qr, pca, *_training, *_prediction} since they are mostly wrappers for computation methods. Results of algorithms are correctly working with pickle:

import daal4py as d4p
import numpy as np
import pickle

x = np.random.uniform(size=(1000, 8))

alg = d4p.qr()
res = alg.compute(x)

p = pickle.dumps(res)
loaded_res = pickle.loads(p)
print((loaded_res.matrixQ == res.matrixQ).all(), (loaded_res.matrixR == res.matrixR).all())

Output: True, True

import daal4py as d4p
import numpy as np
from sklearn.datasets import make_classification
import pickle

x, y = make_classification()

alg = d4p.svm_training()
res = alg.compute(x, y.reshape((y.shape[0], 1)))

p = pickle.dumps(res)
loaded_res = pickle.loads(p)
print((loaded_res.model.SupportVectors == res.model.SupportVectors).all())

Output True

[fschlimb]

What do you mean? Two people asked for the feature. Why is that not obvious?

A very obvious reason is when you write a generic code accepting an algorithm to call, e.g. if the algorithm is a n argument. You cannot let a remote worker execute such a function if the algorithm cannot be pickle'd.

[fschlimb]

Even worse: the current lack of pickle support makes any class/object with a algorithm member unpickle'able.