aim icon indicating copy to clipboard operation
aim copied to clipboard

Aim crashes after trying to add duplicate tag to run

Open feldlime opened this issue 10 months ago • 2 comments
trafficstars

🐛 Bug

When trying to add the same tag to a run, it fails as expected (but the error is not clear at all).
After this, it's impossible to create another run, it fails with SQLite error.

To reproduce

Part 1

run = aim.Run(repo=REPO)
run.add_tag("some_tag")
run.add_tag("some_tag")

This fails with TypeError: __init__() missing 2 required positional arguments: 'params' and 'orig'

Part 2

After this basic run creation

run = aim.Run(repo=REPO)

fails with error

PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.IntegrityError) UNIQUE constraint failed: run_tag.run_id, run_tag.tag_id
[SQL: INSERT INTO run_tag (run_id, tag_id) VALUES (?, ?)]
[parameters: (30, 2)]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

Expected behavior

In Part 1 expected a clear error like the tag is already present in the run and no error after this.

Environment

  • Aim Version: 3.24.0
  • Python version: 3.9
  • pip version --
  • OS (e.g., Linux): Linux
  • Any other relevant information: Running on the remote server

feldlime avatar Jan 02 '25 12:01 feldlime

Hey @feldlime! Thanks for opening the issue. I was just trying out this on my end and it works properly. Can I ask you to upgrade to the latest version (3.27.0) and check again? We have addressed an issue regarding tag addition (#3216) in the latest release, which should also fix the issue you have described.

mihran113 avatar Jan 15 '25 13:01 mihran113

Still happens for me on 3.27.0.

Minimal example (basically same as above):

#!/usr/bin/env python

import aim

run = aim.Run(repo=".")
run.add_tag("some_tag")
run.add_tag("some_tag")
Full log
Traceback (most recent call last):
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
    self.dialect.do_execute(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 942, in do_execute
    cursor.execute(statement, parameters)
sqlite3.IntegrityError: UNIQUE constraint failed: run_tag.run_id, run_tag.tag_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/aim/storage/structured/sql_engine/entities.py", line 233, in add_tag
    session_commit_or_flush(session)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/aim/storage/structured/sql_engine/entities.py", line 31, in session_commit_or_flush
    session.commit()
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2032, in commit
    trans.commit(_to_root=True)
  File "<string>", line 2, in commit
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1313, in commit
    self._prepare_impl()
  File "<string>", line 2, in _prepare_impl
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1288, in _prepare_impl
    self.session.flush()
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4353, in flush
    self._flush(objects)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4488, in _flush
    with util.safe_reraise():
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4449, in _flush
    flush_context.execute()
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 591, in execute
    self.dependency_processor.process_saves(uow, states)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/dependency.py", line 1197, in process_saves
    self._run_crud(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/dependency.py", line 1260, in _run_crud
    connection.execute(statement, secondary_insert)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1416, in execute
    return meth(
           ^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
    return connection._execute_clauseelement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1638, in _execute_clauseelement
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1843, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1983, in _exec_single_context
    self._handle_dbapi_exception(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2352, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
    self.dialect.do_execute(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 942, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: run_tag.run_id, run_tag.tag_id
[SQL: INSERT INTO run_tag (run_id, tag_id) VALUES (?, ?)]
[parameters: (1, 1)]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
    self.dialect.do_execute(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 942, in do_execute
    cursor.execute(statement, parameters)
sqlite3.IntegrityError: UNIQUE constraint failed: run_tag.run_id, run_tag.tag_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/luzmn/tmp/test.py", line 7, in <module>
    run.add_tag("some_tag")
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/aim/sdk/run.py", line 248, in add_tag
    return self.props.add_tag(value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/aim/storage/structured/sql_engine/entities.py", line 237, in add_tag
    session_commit_or_flush(session)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/aim/storage/structured/sql_engine/entities.py", line 31, in session_commit_or_flush
    session.commit()
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2032, in commit
    trans.commit(_to_root=True)
  File "<string>", line 2, in commit
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1313, in commit
    self._prepare_impl()
  File "<string>", line 2, in _prepare_impl
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1288, in _prepare_impl
    self.session.flush()
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4353, in flush
    self._flush(objects)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4488, in _flush
    with util.safe_reraise():
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4449, in _flush
    flush_context.execute()
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 591, in execute
    self.dependency_processor.process_saves(uow, states)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/dependency.py", line 1197, in process_saves
    self._run_crud(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/orm/dependency.py", line 1260, in _run_crud
    connection.execute(statement, secondary_insert)
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1416, in execute
    return meth(
           ^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
    return connection._execute_clauseelement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1638, in _execute_clauseelement
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1843, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1983, in _exec_single_context
    self._handle_dbapi_exception(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2352, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1964, in _exec_single_context
    self.dialect.do_execute(
  File "/home/luzmn/.local/share/mamba/envs/tmp/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 942, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: run_tag.run_id, run_tag.tag_id
[SQL: INSERT INTO run_tag (run_id, tag_id) VALUES (?, ?)]
[parameters: (1, 1)]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

qzed avatar Jan 24 '25 23:01 qzed