The provided lr scheduler `LambdaLR` doesn't follow PyTorch's LRScheduler API.

[benninkcorien]

• pytorch 2.0.0
• python 3.10.10
• cuda 11.7

I'm trying to run the Demo code, but keep getting this error when it tries to call ai.train()
Is this an issue with aitextgen or with the pytorch/lightning package?

pytorch_lightning.utilities.exceptions.MisconfigurationException: 
The provided lr scheduler `LambdaLR` doesn't follow PyTorch's LRScheduler API. 
You should override the  `LightningModule.lr_scheduler_step` hook with your own logic if you are using a custom LR scheduler.

---

The demo script (with some prints added to see where it fails)

from aitextgen.TokenDataset import TokenDataset
from aitextgen.tokenizers import train_tokenizer
from aitextgen.utils import GPT2ConfigCPU
from aitextgen import aitextgen

file_name = "F:\\AITextGen\\myfile.txt"

print("train tokenizer")

train_tokenizer(file_name)
tokenizer_file = "aitextgen.tokenizer.json"
ai = aitextgen(tokenizer_file=tokenizer_file)

print("build dataset for training")
data = TokenDataset(file_name, tokenizer_file=tokenizer_file, block_size=64)

print("Train using my own text")

ai.train(data, batch_size=8, num_steps=50000, generate_every=1000, save_every=1000)

print("generate response")
# Generate text from it!
ai.generate(10, prompt="My Prompt:")

[benninkcorien]

Full error message:

Traceback (most recent call last): File "F:\LanguageCodeNLP\AITextGen\trainwithgpu.py", line 33, in ai.train(data, batch_size=8, num_steps=50000, generate_every=5000, save_every=5000) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\aitextgen\aitextgen.py", line 752, in train trainer.fit(train_model) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 696, in fit self._call_and_handle_interrupt( File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 650, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 735, in _fit_impl results = self._run(model, ckpt_path=self.ckpt_path) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1147, in _run self.strategy.setup(self) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\strategies\single_device.py", line 74, in setup super().setup(trainer) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\strategies\strategy.py", line 153, in setup self.setup_optimizers(trainer) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\strategies\strategy.py", line 141, in setup_optimizers self.optimizers, self.lr_scheduler_configs, self.optimizer_frequencies = _init_optimizers_and_lr_schedulers( File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\core\optimizer.py", line 194, in _init_optimizers_and_lr_schedulers _validate_scheduler_api(lr_scheduler_configs, model) File "C:\Users\Corien\AppData\Local\Programs\Python\Python310\lib\site-packages\pytorch_lightning\core\optimizer.py", line 351, in _validate_scheduler_api raise MisconfigurationException( pytorch_lightning.utilities.exceptions.MisconfigurationException: The provided lr scheduler LambdaLR doesn't follow PyTorch's LRScheduler API. You should override the LightningModule.lr_scheduler_step hook with your own logic if you are using a custom LR scheduler.

[benninkcorien]

          I manage to resolve the error on the google colab by running this:
          !pip install -qq pytorch-lightning==1.7.0 transformers==4.21.3 aitextgen==0.6.0
          Please do let me know if it also solves your issue on colab
          

Originally posted by @analyticray in https://github.com/minimaxir/aitextgen/issues/218#issuecomment-1490176216

That seems to fix things in Colab for me, but not in the local install.

[darrinh]

On Ubuntu 20.04 the above version install results in the same error as the OP.

[darrinh]

The combo !pip install -qq pytorch-lightning==1.7.0 transformers==4.21.3 aitextgen==0.6.0 does not work with python 3.10.6.

[darrinh]

on python 3.11, installing pytorch-lightning==1.7.0 results in:

error: can't find Rust compiler