pynndescent
pynndescent copied to clipboard
Loading indices across various python versions
Hi, this is more a developer question, then a bug report. I want to load classifiers like across different environments:
knn = make_pipeline(
PyNNDescentTransformer(
n_neighbors=self.classifier_dict["n_neighbors"],
n_jobs=settings.n_jobs,
),
KNeighborsClassifier(metric="precomputed", weights=self.classifier_dict["weights"]),
)
knn.fit(train_X, train_Y)
joblib.dump(
knn,
open(
os.path.join(
adata.uns["_save_path_trained_models"],
"scvi_knn_classifier.joblib",
),
"wb",
),
)
Currently they are restricted to a single python minor version (tested for python 3.10, 3.11 and 3.12). All expect different numbers of arguments during loading. The error is:
joblib stored in python 3.10 opened in python 3.12: def _unpickle__CustomPickled(serialized):
"""standard unpickling for _CustomPickled.
Uses NumbaPickler to load.
"""
> ctor, states = loads(serialized)
E TypeError: code() argument 13 must be str, not int
Do you have any suggestion to increase compatibility (when I disable numba.jit, it doesn't allow pickling the classifier)?