save an ICL tuned model

[miraculixx]

How can a fitted model be saved? For example, scikit learn models can generally be exported to disk by using joblib or cloudpickle as noted here

Thanks for the amazing work.

[LennartPurucker]

We had a similar decision on our Discord before, take a look at https://discord.com/channels/1285598202732482621/1316254428654866462 and let me know if it works for you!

[LennartPurucker]

In summary from the Discord discussion, you can save the model with pickle as expected:

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
import pickle
from tabpfn import TabPFNClassifier

# Load data
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

# Train classifier
classifier = TabPFNClassifier(device='cpu')
classifier.fit(X_train, y_train)

# Save the trained classifier to a file
with open('tabpfn_classifier.pkl', 'wb') as f:
    pickle.dump(classifier, f)

# Can be in a separate code
# Load the classifier from the file
with open('tabpfn_classifier.pkl', 'rb') as f:
    loaded_classifier = pickle.load(f)

# Predict
y_pred, p_pred = loaded_classifier.predict(X_test)

[noahho]

I'd keep this open until we've properly documented it @LennartPurucker right?

[LennartPurucker]

Mhm, we are not diverging from the default, so I do not know if we need extra documentation. Nevertheless, we could clarify what saving a fitted TabPFN model even means and the alternatives, such as KV caching.

Let us keep it open for now.

[noahho]

I just implemented an improved workflow that allows you to save a model independent of the checkpoint which means you don't have to save model weights every time and saving is independent of pickle! "Q: How do I save and load a trained TabPFN model?" in the READM

[RodrigoEstrela]

Hello, is this https://arc.net/l/quote/swwdgajw then deprecated? It doesn't work for me. I'm getting model not found on model_loading although I do have it.