File system scheme 'http' not implemented

[wanzysky]

Description

When I tried to run the example about the EN-DE translation, I obtained this error tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'http' not implemented (file: 'http://www.statmt.org/wmt13/training-parallel-commoncrawl.tgz')

Environment information

OS: Linux_x86_64_ubuntu

$ pip freeze | grep tensor
tensor2tensor==1.7.0
tensorboard==1.9.0
tensorflow==1.9.0

$ python -V
Python 3.6.5 :: Anaconda, Inc.

For bugs: reproduction and error logs

➜  tensor2tensor git:(master) ✗ cat prepare.sh
PROBLEM=translate_enfr_wmt32k
MODEL=transformer
HPARAMS=transformer_base_single_gpu

DATA_DIR=$HOME/data/t2t_data
TMP_DIR=$HOME/tmp/t2t_datagen
TRAIN_DIR=./t2t_train/$PROBLEM/$MODEL-$HPARAMS
mkdir -p $DATA_DIR $TMP_DIR $TRAIN_DIR

# Generate data
t2t-datagen \
  --data_dir=$DATA_DIR \
  --tmp_dir=$TMP_DIR \
  --problem=$PROBLEM

# Error logs:
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'http' not implemented (file: 'http://www.statmt.org/wmt13/training-parallel-commoncrawl.tgz')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wanzhaoyi/workspace/anaconda3/bin/t2t-datagen", line 27, in <module>
    tf.app.run()
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/home/wanzhaoyi/workspace/anaconda3/bin/t2t-datagen", line 23, in main
    t2t_datagen.main(argv)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/bin/t2t_datagen.py", line 182, in main
    generate_data_for_registered_problem(problem)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/bin/t2t_datagen.py", line 232, in generate_data_for_registered_problem
    problem.generate_data(data_dir, tmp_dir, task_id)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/text_problems.py", line 294, in generate_data
    self.generate_encoded_samples(data_dir, tmp_dir, split)), paths)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/text_problems.py", line 254, in generate_encoded_samples
    generator = self.generate_samples(data_dir, tmp_dir, dataset_split)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/translate.py", line 55, in generate_samples
    tag))
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/translate.py", line 98, in compile_data
    generator_utils.maybe_download(tmp_dir, compressed_filename, url)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/generator_utils.py", line 219, in maybe_download
    uri, inprogress_filepath, reporthook=download_report_hook)
  File "/home/wanzhaoyi/workspace/anaconda3/lib/python3.6/urllib/request.py", line 289, in urlretrieve
    % (read, size), result)
urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 254984192 out of 918311367 bytes>

[whwhwwhh]

I had the same problem...

[jakezhaojb]

anyway we have the solution to this?

[rzou15]

Hi, I encountered exactly the same problem. Any solutions to share?

[whwhwwhh]

Hi, I encountered exactly the same problem. Any solutions to share?

I was running it on a remote server. The problem was caused by the code could not find the right python package path. After exporting python path, my one works. Hope it helps you.

[princeeaso]

anyway we have the solution to this?

if any pls share