automl
How to fine-tune the TabPFN model?
I am using the TabPFN model for a classification task and would like to fine-tune the already trained pre-trained model to better adapt it to my specific dataset. I have reviewed the documentation and code, but I still have some questions about the specific steps and parameter settings for fine-tuning. Does fine-tuning TabPFN involve gradient updates? How many training epochs are needed? During the fine-tuning process, which parameters are the most important?
Dear setipsh, We have fine-tuning not running in the current code-base. I will tag this Issue with a todo, to re-enable finetuning in our codebase. It was removed when cleaning up our code and improving code quality. Hope to get this in soon!
+1 from me, very interested in having fine tuning available for V2. Many thanks!
Dear @noahho could you mention what happens under the hood when a user calls .train? Also, is the model supposed to be backpropagable if we simply do inference with a torch tensor as input?
Dear setipsh, We have fine-tuning not running in the current code-base. I will tag this Issue with a todo, to re-enable finetuning in our codebase. It was removed when cleaning up our code and improving code quality. Hope to get this in soon!
when this finetunig code be released? we want to try it on our materials dataset
when this finetunig code be released? we want to try it on our materials dataset
Hey, if you are still looking for fine-tuning code, you can check out this repo. This repo is about fine-tuning TabPFN on one downstream tabular dataset.
Fine tuning was recently enabled again! See the examples:
- https://github.com/PriorLabs/TabPFN/blob/main/examples/finetune_classifier.py
- https://github.com/PriorLabs/TabPFN/blob/main/examples/finetune_regressor.py Feel free to open a new issue if you have any trouble.
Fine tuning was recently enabled again! See the examples:
* https://github.com/PriorLabs/TabPFN/blob/main/examples/finetune_classifier.py * https://github.com/PriorLabs/TabPFN/blob/main/examples/finetune_regressor.py Feel free to open a new issue if you have any trouble.
Hi oscarkey,
This might be a dumb question, after finetuning is complete according to the example you've mentioned (specifically the classifier) how do you save the finetuned model and then later use it?
no dumb questions! I think you should be able to use:
tabpfn.model_loading.save_tabpfn_model(my_ft_classifier, "checkpoint_name.pt")
loaded_classifier = TabPFNClassifier(model_path="checkpoint_name.pt")
If this doesn't work, open a new issue and we'll look into it :)