ModuleNotFoundError: No module named 'xLSTM.data'

[mw66]

After:

$ pip install .

do

$ python3 examples/language_modeling.py 
...
ModuleNotFoundError: No module named 'xLSTM.data'

[nicolay-r]

@mw66, I just come over and looking at the related script. I believe the issue is with the https://github.com/muditbhargava66/PyxLSTM/blob/a34c2526aa2646203e6d512ed3186765bbd20b3b/examples/language_modeling.py#L6

where the actulal file is stored there: https://github.com/muditbhargava66/PyxLSTM/blob/main/data/dataset.py So that you have to setup the correct import path, as both dataset and tokenizer are not a part of the xLSTM package.

[mw66]

Since it's a pip package, why not update the package to include these files to be self contained?

[nicolay-r]

I can understand your confusion, but I believe it is an Occam's razor concept. Dataset and Language model feature is not conceptually a part of xLSTM rather just a particular application. So that author mindfully not include them into the pip package. As a result, all the supplements are expected to be manually fixed.

[nicolay-r]

Simply just copy the context from the example script and update the imports

[mw66]

Then at least change

from xLSTM.data import LanguageModelingDataset, Tokenizer 

to be

from data import LanguageModelingDataset, Tokenizer 

i.e. data is not part of the xLSTM module to avoid user confusion.

[nicolay-r]

:ok_hand: I believe @muditbhargava66 will back to when it finds him well

[muditbhargava66]

The issue has been definitively resolved in the latest update, Version 2.0.0.