NameError: name 'IterableWrapper' is not defined

[rkingery]

🐛 Bug

Hello, I am trying to load torchtext datasets to reproduce a couple of the tutorials using the new PyTorch MPS support on Mac. After downloading torchdata I get the following error when trying to load any of the datasets in torchtext.datasets:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [51], in <cell line: 1>()
----> 1 data = torchtext.datasets.IMDB(
      2     '~/data',
      3     split='train'
      4 )

File ~/opt/anaconda3/envs/main/lib/python3.10/site-packages/torchtext/data/datasets_utils.py:193, in _create_dataset_directory.<locals>.decorator.<locals>.wrapper(root, *args, **kwargs)
    191 if not os.path.exists(new_root):
    192     os.makedirs(new_root, exist_ok=True)
--> 193 return fn(root=new_root, *args, **kwargs)

File ~/opt/anaconda3/envs/main/lib/python3.10/site-packages/torchtext/data/datasets_utils.py:155, in _wrap_split_argument_with_fn.<locals>.new_fn(root, split, **kwargs)
    153 result = []
    154 for item in _check_default_set(split, splits, fn.__name__):
--> 155     result.append(fn(root, item, **kwargs))
    156 return _wrap_datasets(tuple(result), split)

File ~/opt/anaconda3/envs/main/lib/python3.10/site-packages/torchtext/datasets/imdb.py:86, in IMDB(root, split)
     81 if not is_module_available("torchdata"):
     82     raise ModuleNotFoundError(
     83         "Package `torchdata` not found. Please install following instructions at `https://github.com/pytorch/data`"
     84     )
---> 86 url_dp = IterableWrapper([URL])
     88 cache_compressed_dp = url_dp.on_disk_cache(
     89     filepath_fn=partial(_filepath_fn, root),
     90     hash_dict={_filepath_fn(root): MD5},
     91     hash_type="md5",
     92 )
     93 cache_compressed_dp = HttpReader(cache_compressed_dp).end_caching(mode="wb", same_filepath_fn=True)

NameError: name 'IterableWrapper' is not defined

My environment screen dump is below. Thanks.

Collecting environment information...
PyTorch version: 1.13.0.dev20220525
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.3.1 (x86_64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2.5)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.4 (main, Mar 31 2022, 03:38:35) [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] torch==1.13.0.dev20220525
[pip3] torchaudio==0.12.0.dev20220525
[pip3] torchdata==0.5.0.dev20220525
[pip3] torchtext==0.13.0.dev20220525
[pip3] torchvision==0.14.0.dev20220525
[conda] numpy                     1.22.4                   pypi_0    pypi
[conda] torch                     1.13.0.dev20220525          pypi_0    pypi
[conda] torchaudio                0.12.0.dev20220525          pypi_0    pypi
[conda] torchdata                 0.5.0.dev20220525          pypi_0    pypi
[conda] torchtext                 0.13.0.dev20220525          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220525          pypi_0    pypi

torchtext version is  0.13.0.dev20220525

[parmeet]

hmm, seem strange. The exception is not raised here which means that torchdata is already available on the system (as also listed in your environment). If that is the case, Iterablewrapper should have been imported here.

[tianyu-z]

I have the same bug

[yushikmr]

I have the same bug

my env:

python=3.8.5
torch==1.12.0
torchtext==0.13.0

[Nayef211]

I also tried reproing but am not able to do so

>>> from torchtext.datasets import WikiText2
>>> train_iter = WikiText2(split='train')

Env:

python==3.10.0
pytorch==1.12.0
torchtext==0.13.0
torchdata==0.4.0

@ejguan would you happen to have any ideas as to what could cause this error to surface? As @parmeet mentioned, torchdata is already available on the system and IterableWrapper has been imported.

[ejguan]

On MacOS, I can't reproduce it either

• nightly releases

torch=1.13.0.dev20220711
torchtext=0.14.0.dev20220707
torchdata=0.5.0.dev20220711

• official releases

torch=1.12.0
torchtext=0.13.0
torchdata=0.4.0

@rkingery Could you please try to use the latest nightly release?

@tianyu-z @yushikmr Could you please check if torchdata is installed with the right version.

As @parmeet mentioned, torchdata is already available on the system and IterableWrapper has been imported.

TBH, I agree this is super strange and I can't reproduce it.

[MadSleep]

You shoud install torchdata, then restart and import torchdata. It worked for me.

[Nayef211]

@rkingery let me know if @MadSleep's suggestion helps fix your issue. Will close out this issue if it does.

[ivaquero]

After installing torchdata, I got another error

ImportError                               Traceback (most recent call last)
/Users/integzz/Documents/blog-bio/pytorch/4-rnn.ipynb Cell 23 in <cell line: 1>()
----> [1](vscode-notebook-cell:/Users/integzz/Documents/blog-bio/pytorch/4-rnn.ipynb#ch0000022?line=0) from torchtext.datasets import IMDB
      [2](vscode-notebook-cell:/Users/integzz/Documents/blog-bio/pytorch/4-rnn.ipynb#ch0000022?line=1) from torch.utils.data.dataset import random_split
      [4](vscode-notebook-cell:/Users/integzz/Documents/blog-bio/pytorch/4-rnn.ipynb#ch0000022?line=3) # Step 1: load and create the datasets

File /opt/homebrew/Caskroom/mambaforge/base/envs/kaggle/lib/python3.10/site-packages/torchtext/__init__.py:12, in <module>
      8 _TEXT_BUCKET = "https://download.pytorch.org/models/text/"
     10 _CACHE_DIR = os.path.expanduser(os.path.join(_get_torch_home(), "text"))
---> 12 from . import data, datasets, functional, models, nn, transforms, utils, vocab
     14 try:
     15     from .version import __version__, git_version  # noqa: F401

File /opt/homebrew/Caskroom/mambaforge/base/envs/kaggle/lib/python3.10/site-packages/torchtext/datasets/__init__.py:3, in <module>
      1 import importlib
----> 3 from .ag_news import AG_NEWS
      4 from .amazonreviewfull import AmazonReviewFull
      5 from .amazonreviewpolarity import AmazonReviewPolarity

File /opt/homebrew/Caskroom/mambaforge/base/envs/kaggle/lib/python3.10/site-packages/torchtext/datasets/ag_news.py:12, in <module>
      6 from torchtext.data.datasets_utils import (
      7     _wrap_split_argument,
      8     _create_dataset_directory,
      9 )
...
----> 5 from torch.utils.data.datapipes.utils.common import check_lambda_fn, DILL_AVAILABLE
      7 if DILL_AVAILABLE:
      8     import dill

ImportError: cannot import name 'check_lambda_fn' from 'torch.utils.data.datapipes.utils.common'

OS: macOS M1 Python: 3.10.5

pytorch==1.12.0 torchdata==0.3.0 torchtext=0.13.0

[ejguan]

@ivaquero

You need torchdata==0.4.0 to work with pytorch 1.12.0

[yushikmr]

I found same error in spite of torchdata==0.4.0 to work with pytorch 1.12.0 #1819

Python: 3.10.5

pytorch==1.12.0
torchdata==0.4.0
torchtext=0.13.0

[ivaquero]

@ivaquero

You need torchdata==0.4.0 to work with pytorch 1.12.0

Unfortunately, there is no torchdata==0.4.0 osx-arm64 build on conda channels.

[ejguan]

@ivaquero I see. I will see if torchdata can provide one for m1 mac. It should be minimum to install torchdata from source by

git clone https://github.com/pytorch/data
cd data
python setup.py install

[MadSleep]

Try to run your code on google colab

[Pythonash]

I found the solution in operating on colab.

You should rerun your notebook as below:

!pip install torchdata
exit()

and re-turn on your runtime and:

import torchtext
import torchdata

Then, your task will be working on well.

[ejguan]

I am closing this Issue because TorchData currently provides pre-assembled binaries for Mac M1 on PyPI and Conda after 0.4.1

You should be able to install torchdata == 0.4.1 and torchtext == 0.13.1 via

• pip3 install torch torchdata torchtext
• conda install pytorch torchdata torchtext -c pytorch

So, this issue should be resolved for Mac M1.