optuna
Improve error messages when given an empty database object
Description
When starting the dashboard for an empty or non-existing database storage, the dashboard crashes when trying to determine the version information. The storage is empty and there is thus no table called version_info.
Traceback (most recent call last):
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1968, in _exec_single_context
self.dialect.do_execute(
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/default.py", line 920, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: version_info
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/storage.py", line 69, in _create_scoped_session
yield session
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/storage.py", line 1070, in _init_version_info_model
version_info = models.VersionInfoModel.find(session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/models.py", line 596, in find
version_info = session.query(cls).one_or_none()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/orm/query.py", line 2773, in one_or_none
return self._iter().one_or_none() # type: ignore
^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/orm/query.py", line 2846, in _iter
result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 2232, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 2127, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1413, in execute
return meth(
^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/sql/elements.py", line 483, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1637, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1987, in _exec_single_context
self._handle_dbapi_exception(
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 2344, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1968, in _exec_single_context
self.dialect.do_execute(
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/sqlalchemy/engine/default.py", line 920, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: version_info
[SQL: SELECT version_info.version_info_id AS version_info_version_info_id, version_info.schema_version AS version_info_schema_version, version_info.library_version AS version_info_library_version
FROM version_info]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/maximilian/.venv/optuna-dashboard/bin/optuna-dashboard", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna_dashboard/_cli.py", line 114, in main
storage = get_storage(args.storage, storage_class=args.storage_class)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna_dashboard/_storage_url.py", line 56, in get_storage
return guess_storage_from_url(storage)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna_dashboard/_storage_url.py", line 67, in guess_storage_from_url
return get_rdb_storage(storage_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna_dashboard/_storage_url.py", line 74, in get_rdb_storage
return RDBStorage(storage_url, skip_compatibility_check=True, skip_table_creation=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/storage.py", line 229, in __init__
self._version_manager = _VersionManager(self.url, self.engine, self.scoped_session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/storage.py", line 1064, in __init__
self._init_version_info_model()
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/storage.py", line 1069, in _init_version_info_model
with _create_scoped_session(self.scoped_session, True) as session:
File "/usr/lib64/python3.11/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File "/home/maximilian/.venv/optuna-dashboard/lib64/python3.11/site-packages/optuna/storages/_rdb/storage.py", line 87, in _create_scoped_session
raise optuna.exceptions.StorageInternalError(message) from e
optuna.exceptions.StorageInternalError: An exception is raised during the commit. This typically happens due to invalid data in the commit, e.g. exceeding max length.
How to Reproduce
- run optuna dashboard
optuna-dashboard sqlite:///db.sqlite3 - optuna fails with
sqlite3.OperationalError: no such table: version_info
Python version
3.11.3
Optuna version
3.1.1
optuna-dashboard version or git revision
0.9.2
Web browser
none
Thank you for your report.
It is actually an intended behavior, not a bug. Do you think we should improve error messages?
I was not aware that this is intended behavior. But I fully understand that perspective. I have some thoughts/comments on that.
If optuna-dashboard is not intended to create storages, it should also not create the file for the respective storage.
Starting the dashboard with optuna-dashboard sqlite:///db.sqlite3 when there was no db.sqlite3 file before, it is created (and empty).
Additionally, as you suggested, the error message could hint, that the storage is completely empty? Or hint the user, that optuna-dashboard can only work on initialized storages?
I was confused that this is an intended behavior, as optuna-dashboard is able to create new studies in the dashboard, and can also work on initialized storages without any studies. I expected to be able to also start optuna-dashboard on empty storages to create my first study via the dashboard as well.
Is there a optuna command to only initialize the storage without creating any studies? This would be helpful to kickstart a new setup.
I see. Thank you for your opinion. Let me change the issue title and remove the bug label.
Is there a optuna command to only initialize the storage without creating any studies? This would be helpful to kickstart a new setup.
optuna command does not provide such a feature. If it's useful for many users, we might be able to consider introducing it to Optuna. As a workaround, you can create an empty database like the following.
$ python -c 'import optuna; optuna.storages.RDBStorage("sqlite:///not-found.db", skip_table_creation=False)'
$ ls not-found.db
not-found.db
I have often been caught in this trap. Most likely also caused: #812
@c-bata I might, can contribute to this, but it is still not clear to: Is the creation of an empty database really intended if the database does not yet exist (e.g. by typo the path)? It is still like that, despite #263
