Incompatible with transformers 4.38

[jowodo]

Hi,

I came across an incompatibility with pip package transformers >=4.38. See https://github.com/huggingface/transformers/issues/29763 I could resolve by restricting version of transformers in pyproject.toml file:

sed -i "s/\^4.11.3/~4.37.2/" pyproject.toml
python3 -m pip install . 

This will probably be fixed upstream, but one should maybe add exempt transformers>=4.38 from the versions. Maybe something like:

sed 's/transformers = "\^4.11.3/transformers = "^4.11.3, !=4.38/' pyproject.toml

[BernhoferM]

Hi,

Thank you for bringing this to my attention.

I'll think about how to best handle this, as restricting it might affect others that run Transformers 4.38 with a compatible PyTorch version (2.2+ seems to work). I personally haven't tested TMbed with anything more recent than PyTorch 2.1.1 but I would assume that it runs just fine with PyTorch 2.2 and up, too.