pyABC
pyABC copied to clipboard
Published
20 hours ago •
ICB-DCM
ICB-DCM
PendingRollbackError reported when using pyabc
Dear @yannikschaelte,
The script reported an error when I ran the pyABC script. Do you have any idea about this error? Is this caused by the SQL storage?
Thank you!
ABC.Sampler INFO: Parallelize sampling on 80 processes.
ABC.History INFO: Start <ABCSMC id=3, start_time=2022-05-05 17:05:58>
ABC INFO: Calibration sample t = -1.
ABC INFO: t: 0, eps: 6.56922806e-02.
PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (builtins.ValueError) Object arrays cannot be saved when allow_pickle=False
[SQL: INSERT INTO summary_statistics (sample_id, name, value) VALUES (?, ?, ?)]
[parameters: [{'name': 'X_2', 'sample_id': 4, 'value': array([0.2358974358974359, 0.11904761904761904, 0.09268292682926829, ...,
0.5618448637316562, 0.3875968992248062, 0.43537414965986393],
dtype=object)}]] (Background on this error at: https://sqlalche.me/e/14/7s2a)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, execution_options, *args, **kw)
1707
-> 1708 context = constructor(
1709 dialect, self, conn, execution_options, *args, **kw
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py in _init_compiled(cls, dialect, connection, dbapi_connection, execution_options, compiled, parameters, invoked_statement, extracted_parameters, cache_hit)
1069 for compiled_params in self.compiled_parameters:
-> 1070 param = [
1071 processors[key](compiled_params[key])
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py in <listcomp>(.0)
1070 param = [
-> 1071 processors[key](compiled_params[key])
1072 if key in processors
~/.local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py in process(value)
1630 def process(value):
-> 1631 return impl_processor(process_param(value, dialect))
1632
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/db_model.py in process_bind_param(self, value, dialect)
39 def process_bind_param(self, value, dialect): # pylint: disable=R0201
---> 40 return to_bytes(value)
41
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/bytes_storage.py in to_bytes(object_)
37 return df_to_bytes(object_)
---> 38 return np_to_bytes(object_)
39
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/numpy_bytes_storage.py in np_to_bytes(arr)
21 f = BytesIO()
---> 22 np.save(f, arr, allow_pickle=False)
23 f.seek(0)
<__array_function__ internals> in save(*args, **kwargs)
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/numpy/lib/npyio.py in save(file, arr, allow_pickle, fix_imports)
527 arr = np.asanyarray(arr)
--> 528 format.write_array(fid, arr, allow_pickle=allow_pickle,
529 pickle_kwargs=dict(fix_imports=fix_imports))
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/numpy/lib/format.py in write_array(fp, array, version, allow_pickle, pickle_kwargs)
659 if not allow_pickle:
--> 660 raise ValueError("Object arrays cannot be saved when "
661 "allow_pickle=False")
ValueError: Object arrays cannot be saved when allow_pickle=False
The above exception was the direct cause of the following exception:
StatementError Traceback (most recent call last)
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/inference/smc.py in wrapped_run(self, *args, **kwargs)
61 # the actual run
---> 62 ret = run(self, *args, **kwargs)
63 finally:
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/inference/smc.py in run(self, minimum_epsilon, max_nr_populations, min_acceptance_rate, max_total_nr_simulations, max_walltime, min_eps_diff)
697 # perform one generation
--> 698 ret = self.run_generation(t=t)
699
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/inference/smc.py in run_generation(self, t)
837 model_names = [model.name for model in self.models]
--> 838 self.history.append_population(
839 t, current_eps, population, n_sim, model_names
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/history.py in append_population(self, t, current_epsilon, population, nr_simulations, model_names)
810
--> 811 self._save_to_population_db(
812 t,
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/history.py in f_wrapper(self, *args, **kwargs)
41 self._make_session()
---> 42 res = f(self, *args, **kwargs)
43 if no_session:
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/history.py in _save_to_population_db(self, t, current_epsilon, nr_simulations, particles_by_model, model_probabilities, model_names)
777 # commit changes
--> 778 self._session.commit()
779
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in commit(self)
1430
-> 1431 self._transaction.commit(_to_root=self.future)
1432
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in commit(self, _to_root)
828 if self._state is not PREPARED:
--> 829 self._prepare_impl()
830
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in _prepare_impl(self)
807 break
--> 808 self.session.flush()
809 else:
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in flush(self, objects)
3362 self._flushing = True
-> 3363 self._flush(objects)
3364 finally:
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in _flush(self, objects)
3502 with util.safe_reraise():
-> 3503 transaction.rollback(_capture_exception=True)
3504
~/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py in __exit__(self, type_, value, traceback)
69 if not self.warn_only:
---> 70 compat.raise_(
71 exc_value,
~/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py in raise_(***failed resolving arguments***)
206 try:
--> 207 raise exception
208 finally:
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in _flush(self, objects)
3462 try:
-> 3463 flush_context.execute()
3464 finally:
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py in execute(self)
455 for rec in topological.sort(self.dependencies, postsort_actions):
--> 456 rec.execute(self)
457
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py in execute(self, uow)
629 def execute(self, uow):
--> 630 util.preloaded.orm_persistence.save_obj(
631 self.mapper,
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py in save_obj(base_mapper, states, uowtransaction, single)
243
--> 244 _emit_insert_statements(
245 base_mapper,
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py in _emit_insert_statements(base_mapper, uowtransaction, mapper, table, insert, bookkeeping)
1236 else:
-> 1237 result = connection._execute_20(
1238 statement,
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py in _execute_20(self, statement, parameters, execution_options)
1619 else:
-> 1620 return meth(self, args_10style, kwargs_10style, execution_options)
1621
~/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py in _execute_on_connection(self, connection, multiparams, params, execution_options, _force)
324 if _force or self.supports_execution:
--> 325 return connection._execute_clauseelement(
326 self, multiparams, params, execution_options
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py in _execute_clauseelement(self, elem, multiparams, params, execution_options)
1486 )
-> 1487 ret = self._execute_context(
1488 dialect,
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, execution_options, *args, **kw)
1713 except BaseException as e:
-> 1714 self._handle_dbapi_exception(
1715 e, util.text_type(statement), parameters, None, None
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py in _handle_dbapi_exception(self, e, statement, parameters, cursor, context)
2031 elif should_wrap:
-> 2032 util.raise_(
2033 sqlalchemy_exception, with_traceback=exc_info[2], from_=e
~/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py in raise_(***failed resolving arguments***)
206 try:
--> 207 raise exception
208 finally:
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, execution_options, *args, **kw)
1707
-> 1708 context = constructor(
1709 dialect, self, conn, execution_options, *args, **kw
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py in _init_compiled(cls, dialect, connection, dbapi_connection, execution_options, compiled, parameters, invoked_statement, extracted_parameters, cache_hit)
1069 for compiled_params in self.compiled_parameters:
-> 1070 param = [
1071 processors[key](compiled_params[key])
~/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py in <listcomp>(.0)
1070 param = [
-> 1071 processors[key](compiled_params[key])
1072 if key in processors
~/.local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py in process(value)
1630 def process(value):
-> 1631 return impl_processor(process_param(value, dialect))
1632
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/db_model.py in process_bind_param(self, value, dialect)
39 def process_bind_param(self, value, dialect): # pylint: disable=R0201
---> 40 return to_bytes(value)
41
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/bytes_storage.py in to_bytes(object_)
37 return df_to_bytes(object_)
---> 38 return np_to_bytes(object_)
39
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/numpy_bytes_storage.py in np_to_bytes(arr)
21 f = BytesIO()
---> 22 np.save(f, arr, allow_pickle=False)
23 f.seek(0)
<__array_function__ internals> in save(*args, **kwargs)
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/numpy/lib/npyio.py in save(file, arr, allow_pickle, fix_imports)
527 arr = np.asanyarray(arr)
--> 528 format.write_array(fid, arr, allow_pickle=allow_pickle,
529 pickle_kwargs=dict(fix_imports=fix_imports))
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/numpy/lib/format.py in write_array(fp, array, version, allow_pickle, pickle_kwargs)
659 if not allow_pickle:
--> 660 raise ValueError("Object arrays cannot be saved when "
661 "allow_pickle=False")
StatementError: (builtins.ValueError) Object arrays cannot be saved when allow_pickle=False
[SQL: INSERT INTO summary_statistics (sample_id, name, value) VALUES (?, ?, ?)]
[parameters: [{'name': 'X_2', 'sample_id': 4, 'value': array([0.2358974358974359, 0.11904761904761904, 0.09268292682926829, ...,
0.5618448637316562, 0.3875968992248062, 0.43537414965986393],
dtype=object)}]]
During handling of the above exception, another exception occurred:
PendingRollbackError Traceback (most recent call last)
<ipython-input-1-099cacb0f3b4> in <module>
368 abc.new(db_path, {"X_2": sum_stat_obs})
369 #history = abc.run(minimum_epsilon=5, max_nr_populations=10)
--> 370 history_3model_improved_2sub_5000_15_multi_5 = abc.run(max_nr_populations=3)
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/inference/smc.py in wrapped_run(self, *args, **kwargs)
63 finally:
64 # close session and store end time
---> 65 self.history.done(end_time=datetime.now())
66 # tell samplers to stop any ongoing processes
67 self.sampler.stop()
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/history.py in f_wrapper(self, *args, **kwargs)
40 if no_session:
41 self._make_session()
---> 42 res = f(self, *args, **kwargs)
43 if no_session:
44 self._close_session()
~/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages/pyabc/storage/history.py in done(self, end_time)
689 end_time = datetime.datetime.now()
690
--> 691 abcsmc = self._session.query(ABCSMC).filter(ABCSMC.id == self.id).one()
692 abcsmc.end_time = end_time
693 self._session.commit()
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py in one(self)
2854
2855 """
-> 2856 return self._iter().one()
2857
2858 def scalar(self):
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py in _iter(self)
2892
2893 statement = self._statement_20()
-> 2894 result = self.session.execute(
2895 statement,
2896 params,
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in execute(self, statement, params, execution_options, bind_arguments, _parent_execute_state, _add_event, **kw)
1689 )
1690 else:
-> 1691 conn = self._connection_for_bind(bind)
1692 result = conn._execute_20(statement, params or {}, execution_options)
1693
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in _connection_for_bind(self, engine, execution_options, **kw)
1530
1531 if self._transaction is not None or self._autobegin():
-> 1532 return self._transaction._connection_for_bind(
1533 engine, execution_options
1534 )
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in _connection_for_bind(self, bind, execution_options)
719
720 def _connection_for_bind(self, bind, execution_options):
--> 721 self._assert_active()
722
723 if bind in self._connections:
~/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py in _assert_active(self, prepared_ok, rollback_ok, deactive_ok, closed_msg)
599 if not deactive_ok and not rollback_ok:
600 if self._rollback_exception:
--> 601 raise sa_exc.PendingRollbackError(
602 "This Session's transaction has been rolled back "
603 "due to a previous exception during flush."
PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (builtins.ValueError) Object arrays cannot be saved when allow_pickle=False
[SQL: INSERT INTO summary_statistics (sample_id, name, value) VALUES (?, ?, ?)]
[parameters: [{'name': 'X_2', 'sample_id': 4, 'value': array([0.2358974358974359, 0.11904761904761904, 0.09268292682926829, ...,
0.5618448637316562, 0.3875968992248062, 0.43537414965986393],
dtype=object)}]] (Background on this error at: https://sqlalche.me/e/14/7s2a)
Best, Duo
Hi @xieduo7 , can you please share the system and version info as described here, and report the sqlalchemy version?
Hi @yannikschaelte,
Thank you for your quick response and sorry for not sharing this information.
Environment
[Report at least the pyABC version (pip show pyabc),
pip show pyabc
Name: pyabc
Version: 0.12.3
Summary: Distributed, likelihood-free ABC-SMC inference
Home-page: https://github.com/icb-dcm/pyabc
Author: The pyABC developers
Author-email: [email protected]
License: BSD-3-Clause
Location: /home/xieduo/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages
Requires: scikit-learn, jabbar, redis, pandas, matplotlib, cloudpickle, click, sqlalchemy, scipy, distributed, numpy, gitpython
Required-by:
python version (which python; python --version),
Python 3.8.13
and system (Ubuntu/Mac/...).]
Linux pbs-master 3.10.0-1160.41.1.el7.x86_64 #1 SMP Tue Aug 31 14:52:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
sqlalchemy version
$ pip show sqlalchemy
Name: SQLAlchemy
Version: 1.4.32
Summary: Database Abstraction Library
Home-page: https://www.sqlalchemy.org
Author: Mike Bayer
Author-email: [email protected]
License: MIT
Location: /home/xieduo/.local/lib/python3.8/site-packages
Requires: greenlet
Required-by: pyabc
Thank you.
Best, Duo
Hi, sorry for not answering, this issue went amiss. It appears you have a ValueError: Object arrays cannot be saved when allow_pickle=False, which can indicate that your summary statistics contain values different from e.g. floats, strings, ints etc, which cannot be correctly stored, see https://pyabc.readthedocs.io/en/latest/api_datastore.html#what-can-be-stored-as-summary-statistics
closing due to non-response. feel free to re-open if issue persists.