sqlite3.OperationalError: database is locked

[abukaj]

While testing for #61 I encountered a following crash:

Traceback (most recent call last):
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 87, in catch_corrupt_db
    return f(self, *a, **kw)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 257, in init_db
    end timestamp, num_cmds integer, remark text)""")
sqlite3.OperationalError: database is locked

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 87, in catch_corrupt_db
    return f(self, *a, **kw)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 257, in init_db
    end timestamp, num_cmds integer, remark text)""")
sqlite3.OperationalError: disk I/O error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jkowalski/anaconda3/envs/Pweave35/bin/pweave", line 11, in <module>
    load_entry_point('Pweave==0.30a1', 'console_scripts', 'pweave')()
  File "/home/jkowalski/Pweave/pweave/scripts.py", line 53, in weave
    pweave.weave(infile, **opts_dict)
  File "/home/jkowalski/Pweave/pweave/__init__.py", line 60, in weave
    doc.weave()
  File "/home/jkowalski/Pweave/pweave/pweb.py", line 192, in weave
    self.run()
  File "/home/jkowalski/Pweave/pweave/pweb.py", line 127, in run
    self.wd
  File "/home/jkowalski/Pweave/pweave/processors/jupyter.py", line 168, in __init__
    super(IPythonProcessor, self).__init__(*args, embed_kernel=embed)
  File "/home/jkowalski/Pweave/pweave/processors/jupyter.py", line 34, in __init__
    km.start_kernel(cwd=path, stderr=open(os.devnull, 'w'))
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/ipykernel/inprocess/manager.py", line 46, in start_kernel
    self.kernel = InProcessKernel(parent=self, session=self.session)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/ipykernel/inprocess/ipkernel.py", line 72, in __init__
    super(InProcessKernel, self).__init__(**traits)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/ipykernel/ipkernel.py", line 46, in __init__
    kernel      = self,
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 460, in __init__
    self.init_history()
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 1564, in init_history
    self.history_manager = HistoryManager(shell=self, parent=self)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 537, in __init__
    **traits)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 229, in __init__
    self.init_db()
  File "<decorator-gen-16>", line 2, in init_db
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 115, in catch_corrupt_db
    self.init_db()
  File "<decorator-gen-16>", line 2, in init_db
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 113, in catch_corrupt_db
    os.rename(self.hist_file, newpath)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jkowalski/.ipython/profile_default/history.sqlite' -> '/home/jkowalski/.ipython/profile_default/history-corrupt.sqlite'

I guess it was due to some kind of race condition since I have been testing #61 paralelly in several shells.

I am opening the issue to preserve information about the problem in case it hit again.

[mpastell]

Seems to be related to https://github.com/ipython/ipython/issues/2426/

[abukaj]

Yes, but there is also FileNotFoundError in the exception handling branch.

I informed IPython developers about the issue (ipython/ipython#10753).

[abukaj]

May we disable writing the history of weaved files?

As an annoying side effect it pollutes IPython history - for example now I have over 8000 sessions like:

%matplotlib inline
from IPython.display import set_matplotlib_formats
set_matplotlib_formats('png', 'pdf', 'svg')
import matplotlib
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
print(123)
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
a = 42
print(a)
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
2 + 2 * 2 ** 2
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
for i in range(3):
    print(i)
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
for i in range(3):
    print(i)
    print(i**2)
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
for i in range(3):
    print(i)
print(1 + 2)
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
for i in range(2):
    print(i)
    if i > 0:
        print("one")
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
for i in range(3):
    print(i)
    if i == 1:
        raise Exception("foo")
print(42)
matplotlib.rcParams.update({"figure.figsize" : (6, 4)})
matplotlib.rcParams.update({"savefig.dpi" : 100})
def f
print(42)

[mpastell]

I already disabled it for commands https://github.com/mpastell/Pweave/commit/5a08b65976da5d4c9ed0d7f984ed96c51bc52a78 . It still creates the file though, I couldn't find a way to disable it completely (apart from editing IPython profile).