readmnist function from the read module fails if the start example is not 0

[mihaelacr]

Trace obtaining by running the spearmint example in the spearmint branch (with the TRAIN and TEST variable being set to 55000 and 5000 respectively).

Traceback (most recent call last):
  File "/data/mcr10/spearmint/spearmint/spearmint/runner.py", line 41, in job_runner
    elif job.language == PYTHON: run_python_job(job)
  File "/data/mcr10/spearmint/spearmint/spearmint/runner.py", line 131, in run_python_job
    result = module.main(job.id, params)
  File "/data/mcr10/project/pydeeplearn/code/spearmint-configs/dbnmnist/mnistdbn.py", line 84, in main
    maxEpochs=params['maxEpochs'][0])
  File "/data/mcr10/project/pydeeplearn/code/spearmint-configs/dbnmnist/mnistdbn.py", line 25, in trainDBN
    readmnist.read(0, TRAIN, digits=None, bTrain=True, path=PATH)
  File "/data/mcr10/project/pydeeplearn/code/read/readmnist.py", line 56, in read
    range(rowsIm*colsIm))
  File "/data/mcr10/project/pydeeplearn/code/read/readmnist.py", line 55, in <lambda>
    vec = map(lambda x: struct.unpack('>B',fImages.read(1))[0],
  File "main.py", line 323, in sigint_handler
    sys.exit(0)

Replacing this code

 trainVectors, trainLabels =\
    readmnist.read(0, TRAIN, digits=None, bTrain=True, path=PATH)

  testVectors, testLabels =\
      readmnist.read(TRAIN, TRAIN + TEST,
                     digits=None, bTrain=True, path=PATH)

With:

  vectors, labels =\
    readmnist.read(0, TRAIN + TEST, digits=None, bTrain=True, path=PATH)
  trainVectors = vectors[0: TRAIN]
  trainLabels = labels[0: TRAIN]
  testVectors = vectors[TRAIN: TRAIN + TEST]
  testLabels = labels[TRAIN: TRAIN + TEST]

solves the problem, which suggests that the code which reads the MNIST digits when the starting example is not 0. Looking at the codebase it also seems that the code was never used with a non zero starting point.

[mihaelacr]

This will be solved by migrating to fuel:

https://github.com/mila-udem/fuel

[ichrakkhaled77]

Hi, can you give me your email ??