ValueError: When passing a model not in PyCaret's model library, the custom_grid parameter must be provided.

[Prashavu]

pycaret version checks

• [X] I have checked that this issue has not already been reported here.

• [X] I have confirmed this bug exists on the latest version of pycaret.

• [ ] I have confirmed this bug exists on the master branch of pycaret (pip install -U git+https://github.com/pycaret/pycaret.git@master).

Issue Description

Valueerror says custom grid to be provided for a model created and saved using Pycaret.

Reproducible Example

from pycaret.datasets import get_data
juice = get_data('juice')
from pycaret.classification import *
exp_name = setup(data = juice,  target = 'Purchase')
lr = create_model('lr')
save_model(lr,'lr1')
tuned_lr = tune_model(lr1,optimize = 'F1',return_train_score=True)

Expected Behavior

Should not expect custom grid as its a model created and saved using optuna. We cannot always create and tune in one go as many are computationally intensive so need to save then and there

Actual Results

/usr/local/lib/python3.7/dist-packages/pycaret/internal/pycaret_experiment/supervised_experiment.py in tune_model(self, estimator, fold, round, n_iter, custom_grid, optimize, custom_scorer, search_library, search_algorithm, early_stopping, early_stopping_max_iters, choose_better, fit_kwargs, groups, return_tuner, verbose, tuner_verbose, return_train_score, **kwargs)
   2253             if custom_grid is None:
   2254                 raise ValueError(
-> 2255                     "When passing a model not in PyCaret's model library, the custom_grid parameter must be provided."
   2256                 )
   2257             estimator_name = self._get_model_name(estimator)

ValueError: When passing a model not in PyCaret's model library, the custom_grid parameter must be provided.

Installed Versions

3.0.0

[celestinoxp]

I think you have an error in your reproducible code...

this line: tuned_lr = tune_model(lr1,optimize = 'F1',return_train_score=True)

you sould change "lr1" to "lr" like this tuned_lr = tune_model(lr,optimize = 'F1',return_train_score=True)

Why? You saved model as lr1 name, but not opened this model as lr1, so in this reproducible code only exists "lr"

To understand better see save_model and load_model section here: https://pycaret.readthedocs.io/en/latest/api/classification.html?highlight=save_model#pycaret.classification.save_model

[Prashavu]

I tried to edit it to correct but was not able to. I saved and loaded the same model using load_model. Hope it clears your doubt

[celestinoxp]

You have to provide more information. Please paste the output of from pycaret import show_versions; show_versions()

If possible, provide your code.

[Prashavu]

[celestinoxp]

I found possible problem. you use python 3.7 and the problem could come from scikit-learn.

You must create a new environment with python 3.8 or python 3.9.

[Prashavu]

System: python: 3.7.13 (default, Apr 24 2022, 01:04:09) [GCC 7.5.0] executable: /usr/bin/python3 machine: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic

PyCaret required dependencies: pip: 21.1.3 setuptools: 57.4.0 pycaret: 3.0.0.rc3 IPython: 7.9.0 ipywidgets: 7.7.1 tqdm: 4.64.0 numpy: 1.21.6 pandas: 1.3.5 jinja2: 2.11.3 scipy: 1.7.3 joblib: 1.1.0 sklearn: 1.0.2 pyod: Installed but version unavailable imblearn: 0.8.1 category_encoders: 2.5.0 lightgbm: 3.3.2 numba: 0.55.2 requests: 2.28.1 matplotlib: 3.5.3 scikitplot: 0.3.7 yellowbrick: 1.4 plotly: 5.10.0 kaleido: 0.2.1 statsmodels: 0.13.2 sktime: 0.11.4 tbats: Installed but version unavailable pmdarima: 1.8.5 psutil: 5.9.1

[Prashavu]

@celestinoxp ok. so anyone using latest Pycaret shoukd strictly use python 3.8 and above?

[celestinoxp]

@celestinoxp OK. então qualquer um que esteja usando o Pycaret mais recente deve usar estritamente o python 3.8 e superior?

Yes. however you should create a new clean environment to avoid package conficts. Note: you shoud only install python 3.8 or python 3.9. Pycaret 3.0rc3 (latest) not support (for now) python 3.10. Mayebe next weeks.

[Prashavu]

@celestinoxp I am using google colab, I am not sure if we can change that without anything breaking /conflicts

[ngupta23]

@Prabhu-kottapu Just to clarify, you can still use python 3.7. All our tests run on 3.7 as of now and pass without issue. See here for more details if needed: https://github.com/pycaret/pycaret/issues/2862

[ngupta23]

Your problem is that the variable lr1 is not defined here in your code. You need to pass the actual model object here which in your code is lr not lr1. lr1 is just the pickle file name of the model saved on your disk.

tuned_lr = tune_model(lr1,optimize = 'F1',return_train_score=True)

[Prashavu]

Like i mention in previous comment. That is not the issue . I forgot to add the load line in example . But I did do it the right way

On Fri, Aug 19, 2022, 4:31 PM Nikhil Gupta @.***> wrote:

Your problem is that the variable lr1 is not defined here in your code. You need to pass the actual model object here which in your code is lr not lr1. lr1 is just the pickle file name of the model saved on your disk.

tuned_lr = tune_model(lr1,optimize = 'F1',return_train_score=True)

— Reply to this email directly, view it on GitHub https://github.com/pycaret/pycaret/issues/2865#issuecomment-1220540486, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADIHHXOHSDMQOPUCJCSXD73VZ5SPJANCNFSM56ZAK2XA . You are receiving this because you authored the thread.Message ID: @.***>

[mydata-ag]

Here's the snippet to reproduce it.

from pycaret.datasets import get_data
from pycaret.classification import *
juice = get_data('juice')
setup(data = juice,  target = 'Purchase', silent=True)
clf = create_model('lr')
save_model(clf, 'lr')
clf_loaded = load_model('lr')
tuned = tune_model(clf_loaded, optimize='F1', return_train_score=True) # throws error

The issue is that load_model returns the whole serialized pipeline, whereas tune_model only expects the classifier.

An ugly workaround would be to use tune_model(clf_loaded[-1]).

[LUS24]

I got the same error:

ValueError: When passing a model not in PyCaret's model library, the custom_grid parameter must be provided.

I'm using pycaret version 2.3.10 with python 3.9.12 in Ubuntu:

Distributor ID: Ubuntu
Description:    Ubuntu 20.04.5 LTS
Release:        20.04
Codename:       focal

The workaround proposed by @mydataag made it work. Thanks!

[github-actions[bot]]

Stale issue message: This issue will be automatically closed by GitHub Actions in 1 week if there is no further activity.