ImportError: No module named cycle

[yucc2018]

first, I run "/dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/run.sh stage1_prepare",that's all right.

then, when I run "/dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/run.sh stage1_train ",I got an error, "ImportError: No module named cycle". However, in my ubuntu server, I can "import cycle" without any question.

root@6bb1cb70814a:~# /dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/run.sh stage1_train  
Running sub-task stage1_train:
First training stage.
Training model /dfsdata/yucc1_data/mini_t3/workingfiles//model_first_1.
Computing model /dfsdata/yucc1_data/mini_t3/workingfiles//model_first_1 with network final_submission.
Traceback (most recent call last):
  File "/dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/code/simplenn_main.py", line 97, in <module>
    experiment = simplenn.Experiment(**options)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 42, in __init__
    self.data = self.preprocess(self.inputs)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 208, in preprocess
    data = map(self.process, inputids)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 182, in process
    data = process.Process(proc, moduleclass=moduleclass, args=self.varargs)(data)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/process/__init__.py", line 76, in __init__
    self.func = util.find_module_function(modulepath=self.modulepath, functionname=self.functionname, defsubpath=moduleclass)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/util.py", line 206, in find_module_function
    modf, path, desc = imp.find_module(modname, paths)
ImportError: No module named cycle
root@6bb1cb70814a:~# python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cycle
>>> 

[danstowell]

Something I don't understand about your "simplenn" install. If you look at simplenn/process/cycle.py it doesn't say import cycle, it says from itertools import cycle.

I think you need to make sure you've installed the correct branch of simplenn? Please check the readme and try that.

[yucc2018]

@danstowell

1. I install "simplenn" with https://jobim.ofai.at/gitlab/gr/simplenn/repository/archive?ref=bird_audio_detection_challenge_2017. Then I run:

/dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/run.sh stage1_train

I will meet the upper error

root@6bb1cb70814a:~# /dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/run.sh stage1_train  
Running sub-task stage1_train:
First training stage.
Training model /dfsdata/yucc1_data/mini_t3/workingfiles//model_first_1.
Computing model /dfsdata/yucc1_data/mini_t3/workingfiles//model_first_1 with network final_submission.
Traceback (most recent call last):
  File "/dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/code/simplenn_main.py", line 97, in <module>
    experiment = simplenn.Experiment(**options)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 42, in __init__
    self.data = self.preprocess(self.inputs)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 208, in preprocess
    data = map(self.process, inputids)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 182, in process
    data = process.Process(proc, moduleclass=moduleclass, args=self.varargs)(data)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/process/__init__.py", line 76, in __init__
    self.func = util.find_module_function(modulepath=self.modulepath, functionname=self.functionname, defsubpath=moduleclass)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/util.py", line 206, in find_module_function
    modf, path, desc = imp.find_module(modname, paths)
ImportError: No module named cycle

2. So, I change to install "simplenn" with https://jobim.ofai.at/gitlab/gr/simplenn. And download the code with "https://jobim.ofai.at/gitlab/gr/simplenn/repository/archive". However, I met the following error:

root@674619ad6684:~/simplenn.git# /dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/run.sh stage1_train
Running sub-task stage1_train:
First training stage.
Training model /dfsdata/yucc1_data/mini_t3/workingfiles//model_first_1.
Computing model /dfsdata/yucc1_data/mini_t3/workingfiles//model_first_1 with network final_submission.
Can not use cuDNN on context None: cannot compile with cuDNN. We got this error:
In file included from /tmp/try_flags_85O764.c:4:0:
/usr/include/cudnn.h:63:26: fatal error: driver_types.h: No such file or directory
compilation terminated.

Mapped name None to device cuda: Tesla P100-PCIE-16GB (0000:2F:00.0)
Epoch   0: train_loss = 0.138
Epoch   1: train_loss = 0.0686
Epoch   2: train_loss = 0.0687
Traceback (most recent call last):
  File "/dfsdata/yucc1_data/mini_t3/bulbul_bird_detection_dcase2018/code/simplenn_main.py", line 101, in <module>
    experiment.train(savefile=savefile, logger=logger)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/experiment.py", line 274, in train
    measures, learning_rate = trainer.train(train_data, val_data)
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/training/trainer.py", line 267, in train
    if not self.learning_rate.update(**measures):
  File "/usr/local/lib/python2.7/dist-packages/simplenn-0.1-py2.7.egg/simplenn/training/learning_rate.py", line 93, in update
    slope,_ = np.linalg.lstsq(np.vstack([np.arange(self.histlen), np.ones(self.histlen)]).T, err[-self.histlen:], rcond=None)[0]
  File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 1953, in lstsq
    0, work, -1, iwork, 0)
TypeError: a float is required

I should install with 1 or 2? And every method has an error.

[danstowell]

Thanks. Definitely 1. However your setup seems a lot like mine, and I'm not sure how this error arises.

[grrrr]

Wait, there seem to be a version mismatch. The cycle module that wants to be imported is a dynamically loaded module sitting in simplenn/process. But, this has been introduced after the bird audio detection challenge. I think that the version of my bulbul code (https://jobim.ofai.at/gitlab/gr/bird_audio_detection_challenge_2017) used here is neither the "official_submission" tag, nor the "portable_submission" tag (which are compatible to the simplenn version 1 you tried), but rather the master branch that has been updated further.

[danstowell]

@grrrr thanks for detecting this. I'm not sure if you mean that the code in this repository needs to change? I see that the version we based on does appear to be your master branch. For other people to be able to dependably reuse bulbul, is it better for us to stick with the older "official_submission", or to update to master and to use a different version of simplenn?

[danstowell]

One possibility is that if we use the latest version instead of the "frozen" version, @yucc2018 can probably use "option 2" (latest simplenn) if they also upgrade their numpy to version 1.14. (The error message reported for option 2 is caused by a change in numpy lstsq API 1.13->1.14.)

@grrrr sounds feasible?

@yucc2018 if you upgrade numpy you may find option 2 works for you. If so I'd be grateful to know.

(BTW sorry slow reply, I was away)

[grrrr]

@danstowell, I would really recommend to use the frozen version, since this is the one that was found to reliably work. The master branch of simplenn is basically a dev branch where i am (or rather was) the only one working and experimenting with. I had troubles lately with excessive memory consumption while iterating data on the master branch and i don't have the capacities to fix that at the moment.

[danstowell]

Thanks @grrrr. What I propose to do now is test a version with cf59be9 reverted ("adhere to current simplenn"), using frozen simplenn, and push that if it seems to work well.

@yucc2018 thanks for your patience - I'll reply here if I push the change and if I think the problem is solved.

[danstowell]

@yucc2018 I've pushed this change now, and I hope this resolves your issue. If you fetch/download the latest version from this github repo, and use the "frozen" simplenn as stated in the readme.