pyhf
pyhf copied to clipboard
Published
20 hours ago •
scikit-hep
scikit-hep
nbclient.exceptions.DeadKernelError from papermill if PYTHONWARNINGS='default'
Summary
When trying to get the Notebook tests to work again, it seems that any notebook that happens to
OS / Environment
GHA runner for `ubuntu-latest`
Steps to Reproduce
On a GHA runner execute
$ pytest --override-ini filterwarnings= tests/test_notebooks.py -k test_xml_importexport
File Upload (optional)
Expected Results
For
$ export PYTHONWARNINGS='default'
$ pytest --override-ini filterwarnings= tests/test_notebooks.py -k test_xml_importexport
to pass on the runner like it does locally
Actual Results
It seems to fail at the
!ls -lavh ../../../validation/xmlimport_input
cell
pyhf Version
dev
Code of Conduct
- [X] I agree to follow the Code of Conduct
@kratsg @lukasheinrich I'm unable to replicate the error that the GHA runners in PR #1841 are seeing when I (on the same branch) run
$ pytest --override-ini filterwarnings= tests/test_notebooks.py -k test_xml_importexport
================================================================= test session starts ==================================================================
platform linux -- Python 3.9.6, pytest-7.1.1, pluggy-1.0.0
Matplotlib: 3.5.1
Freetype: 2.6.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/feickert/Code/GitHub/pyhf, configfile: pyproject.toml
plugins: console-scripts-1.3.1, mock-3.7.0, mpl-0.14.0, requests-mock-1.9.3, benchmark-3.4.1, cov-3.0.0, anyio-3.3.3
collected 12 items / 11 deselected / 1 selected
tests/test_notebooks.py . [100%]
Can one of you try to replicate the GHA runner behavior before I try to bug papermill about this?
Oh. It fails if you add PYTHONWARNINGS='default' into the environment
$ PYTHONWARNINGS='default' pytest --override-ini filterwarnings= tests/test_notebooks.py -k test_xml_importexport &> /tmp/pythonwarnings_log.txt
pythonwarnings_log.txt:
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/requests_mock/contrib/_pytest_plugin.py:15: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
_pytest_version = version.StrictVersion(pytest.__version__)
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/requests_mock/contrib/_pytest_plugin.py:16: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
_pytest29 = _pytest_version >= version.StrictVersion('2.9.0')
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/requests_mock/contrib/_pytest_plugin.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
_pytest30 = _pytest_version >= version.StrictVersion('3.0.0')
============================= test session starts ==============================
platform linux -- Python 3.9.6, pytest-7.1.1, pluggy-1.0.0
Matplotlib: 3.5.1
Freetype: 2.6.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/feickert/Code/GitHub/pyhf, configfile: pyproject.toml
plugins: console-scripts-1.3.1, mock-3.7.0, mpl-0.14.0, requests-mock-1.9.3, benchmark-3.4.1, cov-3.0.0, anyio-3.3.3
collected 12 items / 11 deselected / 1 selected
tests/test_notebooks.py F [100%]
=================================== FAILURES ===================================
____________________________ test_xml_importexport _____________________________
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
msg_id = '00b826eb-55989758c7b1b700d57d9c4d_31186_4'
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
timeout = None
task_poll_output_msg = <Task cancelled name='Task-21' coro=<NotebookClient._async_poll_output_msg() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:756>>
task_poll_kernel_alive = <Task finished name='Task-20' coro=<NotebookClient._async_poll_kernel_alive() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:770> result=None>
async def _async_poll_for_reply(
self,
msg_id: str,
cell: NotebookNode,
timeout: t.Optional[int],
task_poll_output_msg: asyncio.Future,
task_poll_kernel_alive: asyncio.Future,
) -> t.Dict:
assert self.kc is not None
new_timeout: t.Optional[float] = None
if timeout is not None:
deadline = monotonic() + timeout
new_timeout = float(timeout)
while True:
try:
> msg = await ensure_async(self.kc.shell_channel.get_msg(timeout=new_timeout))
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
msg_id = '00b826eb-55989758c7b1b700d57d9c4d_31186_4'
new_timeout = None
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
task_poll_kernel_alive = <Task finished name='Task-20' coro=<NotebookClient._async_poll_kernel_alive() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:770> result=None>
task_poll_output_msg = <Task cancelled name='Task-21' coro=<NotebookClient._async_poll_output_msg() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:756>>
timeout = None
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:730:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = <coroutine object ZMQSocketChannel.get_msg at 0x7fbe9e569940>
async def ensure_async(obj: Union[Awaitable, Any]) -> Any:
"""Convert a non-awaitable object to a coroutine if needed,
and await it if it was not already awaited.
"""
if inspect.isawaitable(obj):
try:
> result = await obj
obj = <coroutine object ZMQSocketChannel.get_msg at 0x7fbe9e569940>
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/util.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <jupyter_client.channels.ZMQSocketChannel object at 0x7fbe9e580f40>
timeout = None
async def get_msg(self, timeout: t.Optional[float] = None) -> t.Dict[str, t.Any]:
"""Gets a message if there is one that is ready."""
assert self.socket is not None
if timeout is not None:
timeout *= 1000 # seconds to ms
> ready = await self.socket.poll(timeout)
E asyncio.exceptions.CancelledError
self = <jupyter_client.channels.ZMQSocketChannel object at 0x7fbe9e580f40>
timeout = None
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/jupyter_client/channels.py:224: CancelledError
During handling of the above exception, another exception occurred:
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
cell_index = 6, execution_count = None, store_history = True
async def async_execute_cell(
self,
cell: NotebookNode,
cell_index: int,
execution_count: t.Optional[int] = None,
store_history: bool = True,
) -> NotebookNode:
"""
Executes a single code cell.
To execute all cells see :meth:`execute`.
Parameters
----------
cell : nbformat.NotebookNode
The cell which is currently being processed.
cell_index : int
The position of the cell within the notebook object.
execution_count : int
The execution count to be assigned to the cell (default: Use kernel response)
store_history : bool
Determines if history should be stored in the kernel (default: False).
Specific to ipython kernels, which can store command histories.
Returns
-------
output : dict
The execution output payload (or None for no output).
Raises
------
CellExecutionError
If execution failed and should raise an exception, this will be raised
with defaults about the failure.
Returns
-------
cell : NotebookNode
The cell which was just processed.
"""
assert self.kc is not None
await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)
if cell.cell_type != 'code' or not cell.source.strip():
self.log.debug("Skipping non-executing cell %s", cell_index)
return cell
if self.skip_cells_with_tag in cell.metadata.get("tags", []):
self.log.debug("Skipping tagged cell %s", cell_index)
return cell
if self.record_timing: # clear execution metadata prior to execution
cell['metadata']['execution'] = {}
self.log.debug("Executing cell:\n%s", cell.source)
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
parent_msg_id = await ensure_async(
self.kc.execute(
cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
)
)
await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
# We launched a code cell to execute
self.code_cells_executed += 1
exec_timeout = self._get_timeout(cell)
cell.outputs = []
self.clear_before_next_output = False
task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
task_poll_output_msg = asyncio.ensure_future(
self._async_poll_output_msg(parent_msg_id, cell, cell_index)
)
self.task_poll_for_reply = asyncio.ensure_future(
self._async_poll_for_reply(
parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
)
)
try:
> exec_reply = await self.task_poll_for_reply
E asyncio.exceptions.CancelledError
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
cell_allows_errors = False
cell_index = 6
exec_timeout = None
execution_count = None
parent_msg_id = '00b826eb-55989758c7b1b700d57d9c4d_31186_4'
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
store_history = True
task_poll_kernel_alive = <Task finished name='Task-20' coro=<NotebookClient._async_poll_kernel_alive() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:770> result=None>
task_poll_output_msg = <Task cancelled name='Task-21' coro=<NotebookClient._async_poll_output_msg() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:756>>
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:949: CancelledError
During handling of the above exception, another exception occurred:
common_kwargs = {'kernel_name': 'python3', 'output_path': PosixPath('/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'), 'progress_bar': False, 'start_timeout': 120}
def test_xml_importexport(common_kwargs):
# Change directories to make users not have to worry about paths to follow example
execution_dir = Path.cwd() / "docs" / "examples" / "notebooks"
> pm.execute_notebook(
execution_dir / "XML_ImportExport.ipynb", cwd=execution_dir, **common_kwargs
)
common_kwargs = {'kernel_name': 'python3', 'output_path': PosixPath('/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'), 'progress_bar': False, 'start_timeout': 120}
execution_dir = PosixPath('/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks')
tests/test_notebooks.py:27:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/papermill/execute.py:107: in execute_notebook
nb = papermill_engines.execute_notebook_with_engine(
cwd = '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks'
engine_kwargs = {}
engine_name = None
input_path = '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb'
kernel_name = 'python3'
language = None
log_output = False
nb = {'cells': [{'cell_type': 'markdown', 'metadata': {'tags': [], 'papermill': {}}, 'source': '# XML Import/Export', 'id':...t_path': '/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'}}, 'nbformat': 4, 'nbformat_minor': 5}
output_path = '/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'
parameters = None
path_parameters = {'pm': {'current_datetime_local': datetime.datetime(2022, 4, 6, 3, 20, 1, 266691), 'current_datetime_utc': datetime.datetime(2022, 4, 6, 8, 20, 1, 266694), 'run_uuid': '9bf111c4-898a-4715-8cb1-4a5cb0899708'}}
prepare_only = False
progress_bar = False
report_mode = False
request_save_on_cell_execute = True
start_timeout = 120
stderr_file = None
stdout_file = None
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/papermill/engines.py:49: in execute_notebook_with_engine
return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
engine_name = None
kernel_name = 'python3'
kwargs = {'input_path': '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb', 'log_output': False, 'output_path': '/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb', 'progress_bar': False, ...}
nb = {'cells': [{'cell_type': 'markdown', 'metadata': {'tags': [], 'papermill': {}}, 'source': '# XML Import/Export', 'id':...t_path': '/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'}}, 'nbformat': 4, 'nbformat_minor': 5}
self = <papermill.engines.PapermillEngines object at 0x7fbeb8968f40>
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/papermill/engines.py:359: in execute_notebook
cls.execute_managed_notebook(nb_man, kernel_name, log_output=log_output, **kwargs)
autosave_cell_every = 30
cls = <class 'papermill.engines.NBClientEngine'>
kernel_name = 'python3'
kwargs = {'input_path': '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb', 'start_timeout': 120, 'stderr_file': None, 'stdout_file': None}
log_output = False
nb = {'cells': [{'cell_type': 'markdown', 'metadata': {'tags': [], 'papermill': {}}, 'source': '# XML Import/Export', 'id':...t_path': '/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'}}, 'nbformat': 4, 'nbformat_minor': 5}
nb_man = <papermill.engines.NotebookExecutionManager object at 0x7fbeb2135970>
output_path = '/tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb'
progress_bar = False
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/papermill/engines.py:418: in execute_managed_notebook
return PapermillNotebookClient(nb_man, **final_kwargs).execute()
cls = <class 'papermill.engines.NBClientEngine'>
execution_timeout = None
final_kwargs = {'input_path': '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb', 'kernel_name': 'python3', 'log': <Logger papermill (INFO)>, 'log_output': False, ...}
kernel_name = 'python3'
kwargs = {'input_path': '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb'}
log_output = False
nb_man = <papermill.engines.NotebookExecutionManager object at 0x7fbeb2135970>
safe_kwargs = {'input_path': '/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb'}
start_timeout = 120
stderr_file = None
stdout_file = None
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/papermill/clientwrap.py:45: in execute
self.papermill_execute_cells()
kwargs = {}
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/papermill/clientwrap.py:72: in papermill_execute_cells
self.execute_cell(cell, index)
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
index = 6
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/util.py:84: in wrapped
return just_run(coro(*args, **kwargs))
args = (<papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>, {'cell_type': 'code', 'execution_count': 4, ...'!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}, 6)
coro = <function NotebookClient.async_execute_cell at 0x7fbeb8973940>
kwargs = {}
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/util.py:62: in just_run
return loop.run_until_complete(coro)
coro = <coroutine object NotebookClient.async_execute_cell at 0x7fbeb220b940>
had_running_loop = False
loop = <_UnixSelectorEventLoop running=False closed=False debug=False>
../../../.pyenv/versions/3.9.6/lib/python3.9/asyncio/base_events.py:642: in run_until_complete
return future.result()
future = <Task finished name='Task-19' coro=<NotebookClient.async_execute_cell() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:864> exception=DeadKernelError('Kernel died')>
new_task = True
self = <_UnixSelectorEventLoop running=False closed=False debug=False>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
cell_index = 6, execution_count = None, store_history = True
async def async_execute_cell(
self,
cell: NotebookNode,
cell_index: int,
execution_count: t.Optional[int] = None,
store_history: bool = True,
) -> NotebookNode:
"""
Executes a single code cell.
To execute all cells see :meth:`execute`.
Parameters
----------
cell : nbformat.NotebookNode
The cell which is currently being processed.
cell_index : int
The position of the cell within the notebook object.
execution_count : int
The execution count to be assigned to the cell (default: Use kernel response)
store_history : bool
Determines if history should be stored in the kernel (default: False).
Specific to ipython kernels, which can store command histories.
Returns
-------
output : dict
The execution output payload (or None for no output).
Raises
------
CellExecutionError
If execution failed and should raise an exception, this will be raised
with defaults about the failure.
Returns
-------
cell : NotebookNode
The cell which was just processed.
"""
assert self.kc is not None
await run_hook(self.on_cell_start, cell=cell, cell_index=cell_index)
if cell.cell_type != 'code' or not cell.source.strip():
self.log.debug("Skipping non-executing cell %s", cell_index)
return cell
if self.skip_cells_with_tag in cell.metadata.get("tags", []):
self.log.debug("Skipping tagged cell %s", cell_index)
return cell
if self.record_timing: # clear execution metadata prior to execution
cell['metadata']['execution'] = {}
self.log.debug("Executing cell:\n%s", cell.source)
cell_allows_errors = (not self.force_raise_errors) and (
self.allow_errors or "raises-exception" in cell.metadata.get("tags", [])
)
await run_hook(self.on_cell_execute, cell=cell, cell_index=cell_index)
parent_msg_id = await ensure_async(
self.kc.execute(
cell.source, store_history=store_history, stop_on_error=not cell_allows_errors
)
)
await run_hook(self.on_cell_complete, cell=cell, cell_index=cell_index)
# We launched a code cell to execute
self.code_cells_executed += 1
exec_timeout = self._get_timeout(cell)
cell.outputs = []
self.clear_before_next_output = False
task_poll_kernel_alive = asyncio.ensure_future(self._async_poll_kernel_alive())
task_poll_output_msg = asyncio.ensure_future(
self._async_poll_output_msg(parent_msg_id, cell, cell_index)
)
self.task_poll_for_reply = asyncio.ensure_future(
self._async_poll_for_reply(
parent_msg_id, cell, exec_timeout, task_poll_output_msg, task_poll_kernel_alive
)
)
try:
exec_reply = await self.task_poll_for_reply
except asyncio.CancelledError:
# can only be cancelled by task_poll_kernel_alive when the kernel is dead
task_poll_output_msg.cancel()
> raise DeadKernelError("Kernel died")
E nbclient.exceptions.DeadKernelError: Kernel died
cell = {'cell_type': 'code', 'execution_count': 4, 'metadata': {'tags': [], 'papermill': {'exception': False, 'start_time': '...e': '!mkdir -p output\n!pyhf json2xml xml_importexport.json --output-dir output\n!ls -lavh output/*', 'id': '36b1477f'}
cell_allows_errors = False
cell_index = 6
exec_timeout = None
execution_count = None
parent_msg_id = '00b826eb-55989758c7b1b700d57d9c4d_31186_4'
self = <papermill.clientwrap.PapermillNotebookClient object at 0x7fbeb2135b20>
store_history = True
task_poll_kernel_alive = <Task finished name='Task-20' coro=<NotebookClient._async_poll_kernel_alive() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:770> result=None>
task_poll_output_msg = <Task cancelled name='Task-21' coro=<NotebookClient._async_poll_output_msg() done, defined at /home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:756>>
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/nbclient/client.py:953: DeadKernelError
----------------------------- Captured stderr call -----------------------------
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/ipykernel/iostream.py:9: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import lock_held as import_lock_held
Assertion failed: ok (src/mailbox.cpp:99)
------------------------------ Captured log call -------------------------------
INFO papermill:execute.py:84 Input Notebook: /home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb
INFO papermill:execute.py:85 Output Notebook: /tmp/pytest-of-feickert/pytest-38/test_xml_importexport0/output.ipynb
INFO papermill:execute.py:88 Working directory: /home/feickert/Code/GitHub/pyhf/docs/examples/notebooks
INFO papermill:clientwrap.py:44 Executing notebook with kernel: python3
ERROR papermill:client.py:808 Kernel died while waiting for execute reply.
=============================== warnings summary ===============================
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/flatbuffers/compat.py:19
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:23
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:24
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:25
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:28
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
if hasattr(pil_image, 'HAMMING'):
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:29
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:30
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
if hasattr(pil_image, 'BOX'):
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:31
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:33
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
if hasattr(pil_image, 'LANCZOS'):
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:34
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:57
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:57: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if (distutils.version.LooseVersion(tf.__version__) <
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:58
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:58: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
distutils.version.LooseVersion(required_tensorflow_version)):
../../../.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/ansiwrap/core.py:11
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/ansiwrap/core.py:11: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/textwrap3.py' mode='r' encoding='utf-8'>
a_textwrap = imp.load_module('a_textwrap', *imp.find_module('textwrap3'))
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/test_notebooks.py::test_xml_importexport
/home/feickert/.pyenv/versions/pyhf-dev-CPU/lib/python3.9/site-packages/traitlets/config/configurable.py:85: DeprecationWarning: Passing unrecognized arguments to super(PapermillNotebookClient).__init__(input_path='/home/feickert/Code/GitHub/pyhf/docs/examples/notebooks/XML_ImportExport.ipynb').
object.__init__() takes exactly one argument (the instance to initialize)
This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
super(Configurable, self).__init__(**kwargs)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
----------- coverage: platform linux, python 3.9.6-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing
-----------------------------------------------------------------------------------
src/pyhf/__init__.py 23 4 6 3 76% 43, 48, 50, 52
src/pyhf/_version.py 2 0 0 0 100%
src/pyhf/cli/__init__.py 9 1 0 0 89% 13
src/pyhf/cli/cli.py 30 2 2 1 91% 18-19
src/pyhf/cli/complete.py 13 6 0 0 54% 17-25
src/pyhf/cli/infer.py 84 51 34 0 28% 83-130, 193-246
src/pyhf/cli/patchset.py 68 40 8 0 37% 42-59, 82-98, 114-125, 137-144
src/pyhf/cli/rootio.py 51 21 4 1 56% 38-39, 50-52, 64-87
src/pyhf/cli/spec.py 148 100 38 0 26% 63-163, 192-209, 256-272, 301-318, 350-366, 396-407
src/pyhf/compat.py 34 15 20 2 50% 11, 47-53, 97-102, 110
src/pyhf/constraints.py 134 115 42 0 11% 11, 16-83, 86-91, 98, 108-125, 136-145, 150-206, 209-214, 221, 231-249, 260-269
src/pyhf/contrib/__init__.py 0 0 0 0 100%
src/pyhf/contrib/cli.py 26 10 4 0 53% 13, 29, 64-73
src/pyhf/contrib/utils.py 44 44 16 0 0% 3-129
src/pyhf/contrib/viz/__init__.py 0 0 0 0 100%
src/pyhf/contrib/viz/brazil.py 65 65 34 0 0% 2-372
src/pyhf/events.py 66 31 10 0 46% 19, 28, 35-36, 42-49, 58-65, 68-73, 76, 79, 82, 85, 115-119, 185, 193
src/pyhf/exceptions/__init__.py 44 16 4 0 58% 28, 59-72, 173-177
src/pyhf/infer/__init__.py 39 31 20 0 14% 9-15, 153-204, 213
src/pyhf/infer/calculators.py 147 109 22 0 24% 30, 76-82, 110-111, 132-133, 168-173, 198-199, 276-284, 315-328, 363-424, 462-467, 507-508, 546-547, 595-596, 657-658, 704-711, 753-817, 857-862, 902-920, 951-960
src/pyhf/infer/intervals.py 20 13 10 0 23% 10, 14-15, 59-75
src/pyhf/infer/mle.py 27 19 8 0 23% 9, 53, 57-59, 117-131, 191-202
src/pyhf/infer/test_statistics.py 60 46 26 0 16% 13, 26-35, 47-60, 130-139, 225-234, 307-316, 397-406, 476-496
src/pyhf/infer/utils.py 19 9 0 0 53% 15, 32-33, 69, 105-113
src/pyhf/interpolators/__init__.py 30 20 8 0 26% 5-18, 31-42, 49
src/pyhf/interpolators/code0.py 52 37 6 0 26% 29-42, 45-50, 53-58, 62-86, 91-97, 100, 103-104
src/pyhf/interpolators/code1.py 58 42 6 0 25% 30-48, 51-62, 65-77, 81-102, 107-113, 116, 119-120
src/pyhf/interpolators/code2.py 62 47 8 0 21% 33-49, 52-60, 63-68, 72-119, 124-132, 135, 138-139
src/pyhf/interpolators/code4.py 86 70 12 0 16% 36-131, 134-148, 153-168, 172-237, 282-356, 359-360, 363-364
src/pyhf/interpolators/code4p.py 67 52 8 0 20% 22-35, 38-47, 50-55, 59-111, 120-132, 135, 138-139
src/pyhf/mixins.py 27 23 10 0 11% 17-51
src/pyhf/modifiers/__init__.py 11 1 0 0 91% 36
src/pyhf/modifiers/histosys.py 80 61 26 0 18% 13, 29-31, 34-38, 41-56, 62-93, 103-141, 144-150, 159-173
src/pyhf/modifiers/lumi.py 52 36 14 0 24% 10, 27-29, 32-34, 37-48, 54, 62-79, 82-89, 96-106
src/pyhf/modifiers/normfactor.py 54 38 14 0 24% 10, 25-27, 30-32, 35-46, 52, 60-79, 82-91, 98-115
src/pyhf/modifiers/normsys.py 68 51 20 0 19% 11, 27-29, 32-39, 42-58, 64, 75-111, 114-123, 132-146
src/pyhf/modifiers/shapefactor.py 65 47 22 0 21% 12, 27-29, 32-34, 37-48, 54, 129-182, 185-197, 204-219
src/pyhf/modifiers/shapesys.py 92 72 36 0 16% 14-25, 42-44, 47-49, 52-66, 72-95, 103-145, 148-167, 172-181, 188-204
src/pyhf/modifiers/staterror.py 105 84 42 0 14% 14-15, 32-34, 37-39, 42-53, 56-124, 133-172, 175-193, 198-207, 210-223
src/pyhf/optimize/__init__.py 22 4 6 1 75% 29-35
src/pyhf/optimize/common.py 44 39 22 0 8% 21-28, 38-58, 109-157
src/pyhf/optimize/mixins.py 63 49 24 1 17% 28, 43-66, 75-112, 159-199
src/pyhf/optimize/opt_jax.py 24 24 4 0 0% 3-80
src/pyhf/optimize/opt_minuit.py 59 44 20 0 19% 52-75, 105-141
src/pyhf/optimize/opt_numpy.py 11 11 2 0 0% 3-32
src/pyhf/optimize/opt_pytorch.py 18 18 2 0 0% 3-42
src/pyhf/optimize/opt_scipy.py 29 17 12 0 29% 42, 72-93
src/pyhf/optimize/opt_tflow.py 18 18 2 0 0% 2-43
src/pyhf/parameters/__init__.py 6 1 0 0 83% 21
src/pyhf/parameters/paramsets.py 66 44 13 0 28% 15, 20-27, 33-35, 40-44, 50-54, 59-60, 65-66, 71-76, 79-82, 87-91, 94-97
src/pyhf/parameters/paramview.py 41 29 8 0 24% 13, 17-32, 36-52, 62-84, 87-90, 99-103
src/pyhf/parameters/utils.py 31 27 16 0 9% 7, 11-70
src/pyhf/patchset.py 95 54 16 0 37% 17, 43-44, 49, 54, 59, 63-65, 69-71, 163-201, 206, 211, 216, 221, 226, 231, 236, 240-242, 255-260, 271, 280, 295-298, 317-318
src/pyhf/pdf.py 319 256 108 0 15% 26, 31-41, 45-56, 61-62, 65-75, 78-95, 111-187, 207-229, 235-237, 243-245, 262-265, 282-285, 303, 320-330, 348, 378-382, 397-404, 407-419, 426-451, 461, 476-488, 502-503, 510-531, 534-535, 545, 548-549, 563, 566-579, 608-631, 659-699, 714-716, 719, 724, 737-739, 752-756, 770, 784, 797-808, 822-850, 864-865
src/pyhf/probability.py 62 37 10 0 35% 8, 25, 35, 48, 68-70, 87, 109-112, 130, 155-156, 185-188, 224-226, 236, 250, 260-261, 274, 287-289, 293-299
src/pyhf/readxml.py 160 27 83 15 77% 30, 71-74, 84, 85->87, 128, 132-144, 163, 171-197, 209, 210->212, 232, 287->289, 290, 293->284, 297, 326-329, 376
src/pyhf/schema/__init__.py 23 6 0 0 74% 19, 74-75, 78, 87, 94
src/pyhf/schema/loader.py 21 2 4 2 84% 12, 35
src/pyhf/schema/validator.py 14 2 0 0 86% 40-41
src/pyhf/schema/variables.py 8 1 2 1 80% 8
src/pyhf/simplemodels.py 13 6 0 0 54% 9, 51-78, 114-141, 148
src/pyhf/tensor/__init__.py 34 6 8 1 83% 21-22, 34-35, 39->exit, 47-48
src/pyhf/tensor/common.py 52 42 20 0 14% 18-29, 32-38, 41-51, 54-63, 70-76, 90-97
src/pyhf/tensor/jax_backend.py 141 76 2 0 45% 18, 22, 28-29, 34-35, 39, 47-48, 91, 112, 132, 154, 178, 181-186, 189, 192, 195, 198, 222-230, 233, 236, 239, 242-251, 254-263, 266, 269, 272, 275, 278, 317, 320, 323, 337, 359, 362, 365, 385, 405, 408-410, 452-454, 460-464, 495, 519, 540, 564, 591, 616
src/pyhf/tensor/manager.py 69 27 42 14 50% 34, 80-82, 84->101, 86, 90, 96-97, 102->113, 104, 108, 114-116, 121-141, 154->169, 162, 163->166, 170, 171->174
src/pyhf/tensor/numpy_backend.py 135 74 2 0 45% 14, 17, 20-21, 26-27, 30, 33-34, 77, 97, 117, 139, 163, 166-171, 174, 177, 180, 183, 206-214, 217, 220, 223, 226-235, 238-247, 250, 253, 256, 259, 262, 300, 303, 306, 320, 342, 345, 348, 368, 387, 390, 432-434, 440-444, 475, 499, 520, 544, 572, 597
src/pyhf/tensor/pytorch_backend.py 130 77 8 0 38% 30, 52, 72, 92, 116, 119-124, 146, 170, 193-201, 204, 207, 210, 213, 233, 236, 243, 246, 249-258, 261-270, 273, 276, 279, 282, 285, 326, 329, 332, 346, 349, 372-385, 399, 403, 446, 451-456, 484-489, 519-520, 543, 567, 594, 619
src/pyhf/tensor/tensorflow_backend.py 148 97 14 0 31% 50-54, 75, 96, 120-127, 152, 155-160, 163-170, 173, 176, 179, 204-221, 224, 231, 238, 241-250, 253-262, 265, 268, 271, 274, 295, 298, 301, 304, 344, 349, 376, 390, 416-424, 441, 470-471, 514-515, 544-547, 576-579, 604-607, 630-632, 657-660, 688, 716
src/pyhf/utils.py 38 23 6 1 36% 13, 24, 28-31, 38-41, 72-84, 103-109
src/pyhf/workspace.py 217 182 140 0 10% 23, 42-73, 92-96, 117-140, 159-182, 203-214, 233-279, 302-321, 327-329, 333, 337, 360-390, 423-442, 461-473, 523-626, 656-662, 691-696, 736-768, 782-796, 813-839
src/pyhf/writexml.py 144 144 62 0 0% 1-312
-----------------------------------------------------------------------------------
TOTAL 4317 2893 1198 43 28%
Coverage HTML written to dir htmlcov
Coverage XML written to file coverage.xml
=========================== short test summary info ============================
FAILED tests/test_notebooks.py::test_xml_importexport - nbclient.exceptions.D...
================ 1 failed, 11 deselected, 14 warnings in 8.17s =================