"cannot open db" while running MNIST.ipynb

[henryguyu]

The db could be found by code in cell 2, "lmdb test db found! Looks like you ran this before, so we need to cleanup those old files... training data folder:/home/henry/caffe2_notebooks/tutorial data/mnist workspace root folder:/home/henry/caffe2_notebooks/tutorial files/tutorial_mnist"

But got code run error in cell14, "RuntimeError: [enforce fail at db.h:206] db_. Cannot open db: /home/henry/caffe2_notebooks/tutorial_data/mnist/mnist-train-nchw-lmdb of type lmdb Error from operator: ..."

[SteveCruz]

Just in order to be sure, do you get the message

lmdb test db found!
Looks like you ran this before, so we need to cleanup those old files...
training data folder:/home/henry/caffe2_notebooks/tutorial data/mnist
workspace root folder:/home/henry/caffe2_notebooks/tutorial files/tutorial_mnist"

or

lmdb train db found!  <--- Is this one missing in your case?
lmdb test db found!
Looks like you ran this before, so we need to cleanup those old files...
training data folder:/home/henry/caffe2_notebooks/tutorial data/mnist
workspace root folder:/home/henry/caffe2_notebooks/tutorial files/tutorial_mnist"

Further, have you manually checked that each of the two folders contain lmdb files?

[henryguyu]

thanks really so much for your response, the issue is killing me. I did get the "lmdb train db found!" And I had manually checked each of th two folders, the lmdb files are there.

20180315.1520.zip

[henryguyu]

[henryguyu]

[henryguyu]

[henryguyu]

@SteveCruz Steve, I uploaded the .ipynb file in zip, and uploaded the screen shots, can you please take a look of them and pull me out of the issue? I am using ubuntu-16.04.3-desktop-amd64 in Oracle VM VirtualBox, can't it cause the problem?

[SteveCruz]

I downloaded your script and tested it on my Ubuntu machine. I changed

data_uint8, label = brew.db_input(
        model,
        blobs_out=["data_uint8", "label"],
        batch_size=batch_size,
        db=db,
        db_type=db_type,
    )

to

data_uint8, label = model.TensorProtosDBInput(
        [],
        ["data_uint8", "label"],
        batch_size=batch_size,
        db=db,
        db_type=db_type,
    )

because I do not have the latest version of Caffe2 on my machine. However, after the change, the script worked perfectly fine for me. Can you try to implement the same changes?

Further, does your VM and/or Jupyter Notebook have read access to the location where your MNIST LMDB is stored?

[henryguyu]

I implemented the code change, but still got the same error. I double checked the read access to the location, it should work. I even tried "sudo ipython notebook" to ensure the access work, but got the same error again. I pasted the screen shot of the LMDB files in last response to you, can you do me a favour to check the sizes of them are same to yours? I unzipped them from the zip file, it should be good, but just want to ensure there is no format error of them or I downloaded the wrong one.

Thanks again for your kind support! @SteveCruz

[SteveCruz]

Here is a screen shot of my LMDB files:

[henryguyu]

totally same, hmm, quite strange, :( I will take a try on ubuntu-17.10.1-desktop-amd64 tomorrow. what's version of your ubuntu? @SteveCruz, thanks!

[SteveCruz]

I use Ubuntu version 16.04.

I am sorry that I could not help to solve your problem!

[henryguyu]

@SteveCruz np, thanks so much indeed! I suppose it may be caused by the C++ code not being well compiled. As the python codes I debuged looks good.

[pjh5]

Can you give your lmdb and protobuf versions? As well as PYTHONPATH and LD_LIBRARY_PATH ? It seems like you're not using Anaconda, are you using virtualenv or virtualenvwrapper?

[miaoshuyu]

I have the same problem as you.And it confused me much time.I hope someone could help me solve it

[henryguyu]

@pjh5 I followed Caffe2 install on Ubuntu with "Build From Source", https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=compile lmdb version is: protobuf version is: PYTHONPATH and LD_LIBRARY_PATH is: I don't remember I use Anaconda, virtualenv or virtualenvwrapper.

[henryguyu]

@pjh5 forgot to thank you for addressing the issue. I noticed I used sudo to run ipython notebook, but missed setting of PYTHONPATH and LD_LIBRARY_PATH for root, I added, but still same error.

[henryguyu]

@pjh5 I can give my ubuntu access if you would like to take a look there. if you want, please leave your email here, I will send hostname/port and id/password to you via email. Thanks!

[pjh5]

@henryguyu does the file /usr/local/lib/libcaffe2.so and the directory /usr/local/lib/python2.7/site-packages/caffe2 both exist?

Is which python /usr/bin/python or /usr/local/bin/python?

Your PYTHONPATH should not have two colons : in a row, and shouldn't have /home/henry/caffe2/build in it either. It probably doesn't need /usr/local to be in it either.

LD_LIBRARY_PATH should not have trailing or leading colons.

[henryguyu]

@pjh5 /usr/local/lib/libcaffe2.so exists. /usr/local/lib/python2.7/site-packages/caffe2 does not exist, nothing under folder site-packages.

I fixed the colon issue, but got the same error again. I even tried to remove PYTHONPATH and LD_LIBRARY_PATH, but the error was same, interesting. Are PYTHONPATH and LD_LIBRARY_PATH useless?

[henryguyu]

@pjh5 from the runtime error trace log, it invoked:

[pjh5]

@henryguyu can you paste that output as text or replace all of the '\n' characters with new lines?

For an explanation of what PYTHONPATH does, see https://caffe2.ai/docs/faq.html#why-do-i-get-import-errors-in-python-when-i-try-to-use-caffe2

[pjh5]

Note: @miaoshuyu is also getting this on "caffe2+anaconda3+cmake3+vs2017"

[henryguyu]

@pjh5 just copied it as below RuntimeError: [enforce fail at db.h:206] db_. Cannot open db: /home/henry/caffe2_notebooks/tutorial_data/mnist/mnist-train-nchw-lmdb of type lmdb Error from operator: output: "dbreader_/home/henry/caffe2_notebooks/tutorial_data/mnist/mnist-train-nchw-lmdb" name: "" type: "CreateDB" arg { name: "db_type" s: "lmdb" } arg { name: "db" s: "/home/henry/caffe2_notebooks/tutorial_data/mnist/mnist-train-nchw-lmdb" } debug_info: " File "", line 1, in File "/usr/lib/python2.7/dist-packages/IPython/kernel/zmq/kernelapp.py", line 469, in main app.start() File "/usr/lib/python2.7/dist-packages/IPython/kernel/zmq/kernelapp.py", line 459, in start ioloop.IOLoop.instance().start() File "/usr/lib/python2.7/dist-packages/zmq/eventloop/ioloop.py", line 162, in start super(ZMQIOLoop, self).start() File "/usr/lib/python2.7/dist-packages/tornado/ioloop.py", line 866, in start handler_func(fd_obj, events) File "/usr/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper return fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events self._handle_recv() File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv self._run_callback(callback, msg) File "/usr/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback callback(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper return fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/IPython/kernel/zmq/ipkernel.py", line 281, in dispatcher return self.dispatch_shell(stream, msg) File "/usr/lib/python2.7/dist-packages/IPython/kernel/zmq/ipkernel.py", line 245, in dispatch_shell handler(stream, idents, msg) File "/usr/lib/python2.7/dist-packages/IPython/kernel/zmq/ipkernel.py", line 389, in execute_request shell.run_cell(code, store_history=store_history, silent=silent) File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2741, in run_cell interactivity=interactivity, compiler=compiler) File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2827, in run_ast_nodes if self.run_code(code): File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2883, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 6, in db_type='lmdb') File "", line 8, in AddInput db_type=db_type, File "/usr/local/lib/python2.7/dist-packages/caffe2/python/model_helper.py", line 452, in TensorProtosDBInput self, blob_out, batch_size, db, db_type, **kwargs) File "/usr/local/lib/python2.7/dist-packages/caffe2/python/helpers/db_input.py", line 29, in db_input db_type=db_type, File "/usr/local/lib/python2.7/dist-packages/caffe2/python/core.py", line 2047, in op_type, *args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/caffe2/python/core.py", line 2024, in _CreateAndAddToSelf op = CreateOperator(op_type, inputs, outputs, **kwargs) "

[henryguyu]

@pjh5 by debug, the script failed at invoking below function

and the variable of args is: <type 'tuple'>: ('\n\x10mnist_train_init\x12\xfa\n\x12Odbreader_/home/henry/caffe2_notebooks/tutorial_data/mnist/mnist-train-nchw-lmdb\x1a\x00"\x08CreateDB*\x0f\n\x07db_type"\x04lmdb*L\n\x02db"F/home/henry/caffe2_notebooks/tutorial_data/mnist/mnist-train-nchw-lmdbR\xbb\t File "/home/henry/eclipse/plugins/org.python.pydev.core_6.3.2.201803171248/pysrc/pydevd.py", line 1621, in \n main()\n File "/home/henry/eclipse/plugins/org.python.pydev.core_6.3.2.201803171248/pysrc/pydevd.py", line 1615, in main\n globals = debugger.run(setup['file'], None, None, is_module)\n File "/home/henry/eclipse/plugins/org.python.pydev.core_6.3.2.201803171248/pysrc/pydevd.py", line 1022, in run\n pydev_imports.execfile(file, globals, locals) # execute the script\n File "/home/henry/eclipse-workspace/20180319.1857/caffe2/MNIST.py", line 319, in \n db_type='lmdb')\n File "/home/henry/eclipse-workspace/20180319.1857/caffe2/MNIST.py", line 144, in AddInput\n db_type=db_type...

[henryguyu]

is there way to let the function give more error information? It just threw an exception but with little useful information.

[henryguyu]

@pjh5 btw, I can run Basic demo under /caffe2/python/tutorials successfully. I think the only difference is the lmdb, right?

[pjh5]

Are people still getting this on the latest code? (the latest code is now on pytorch, at https://github.com/pytorch/pytorch)

If so, read https://caffe2.ai/docs/faq.html#null__how-do-python-installations-work and https://caffe2.ai/docs/faq.html#how-can-i-find-what-dependencies-my-caffe2-library-or-other-library-has and check that:

1. Your python installation is correct. which python and which pip and which conda all point to the same bin folder. PYTHONPATH does not point to another python installation or to build folder
2. lmdb is installed correctly into this python
3. Caffe2 is installed correctly into this python, and has a dependency on lmdb, and can find this lmdb

[henryguyu]

@pjh5 not work yet, learn and practice pytorch now. btw, why caffe2 is on pytorch now? will follow your guide to check the settings, thanks again! almost lost confidence to solve the problem.

[pjh5]

@henryguyu see #2439 for why Caffe2 is on pytorch now.

[lingeo]

does anyone have coped with this problem? I am also facing this bug and i do not find a solution, though i have tried all the methods above.