The Future of torchtext

[lordsoffallen]

❓ Questions and Help

Description

As of September 2023 development efforts on torchtext has been stopped. I am wondering what's the future plans in this regard. To opt in for hugging face libraries such as tokenizers? Currently without using the torchtext library it's not really unclear how to work on simple task like text classfication where we don't use a LLM. I can do the preprocessing in spacy and connect to pytorch but somehow it feels different. I'd prefer to do all in pytorch but so far it doesn't seem possible. I didn't invest time into torchtext since so far there is no future for this library and also tutorials just don't work. Perhaps an update/pointers would be nice.

Thanks in advance

[joecummings]

Hi @lordsoffallen - unfortunately, TorchText is no longer actively maintained by PyTorch.

For NLP model implementations, data transforms, and datasets, I recommend HuggingFace's Transformers, Tokenizers, and Datasets libraries.