hypothesis
hypothesis copied to clipboard
HypothesisWorks
Tests fail with StopTest (OVERRUN) when generating a random integer (strategies.randoms)
I originally thought this is the same issue as #3874, but even after upgrading to hypothesis-6.102.4, my tests sometimes fail as described in $title.
Run log from the last failure:
nox > python -m pytest --exitfirst
==================================================== test session starts ====================================================
platform linux -- Python 3.12.3, pytest-8.2.1, pluggy-1.5.0
rootdir: /home/jstanek/personal/project/shoji
configfile: pyproject.toml
plugins: hypothesis-6.102.4, anyio-4.3.0, trio-0.8.0
collected 12 items
tests/test_management.py ........ [ 66%]
tests/test_testing_helper.py .. [ 83%]
tests/util/test_io.py F
========================================================= FAILURES ==========================================================
__________________________________________ test_reader_receive_line_reads_all_data __________________________________________
+ Exception Group Traceback (most recent call last):
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/runner.py", line 341, in from_call
| result: Optional[TResult] = func()
| ^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/runner.py", line 241, in <lambda>
| lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
| return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
| return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 182, in _multicall
| return outcome.get_result()
| ^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_result.py", line 100, in get_result
| raise exc.with_traceback(exc.__traceback__)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_call
| yield from thread_exception_runtest_hook()
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/threadexception.py", line 63, in thread_exception_runtest_hook
| yield
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_call
| yield from unraisable_exception_runtest_hook()
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 65, in unraisable_exception_runtest_hook
| yield
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
| yield from self._runtest_for(item, "call")
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/logging.py", line 833, in _runtest_for
| yield
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/capture.py", line 878, in pytest_runtest_call
| return (yield)
| ^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
| return (yield)
| ^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
| res = hook_impl.function(*args)
| ^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/runner.py", line 173, in pytest_runtest_call
| item.runtest()
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/python.py", line 1632, in runtest
| self.ihook.pytest_pyfunc_call(pyfuncitem=self)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
| return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
| return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
| raise exception.with_traceback(exception.__traceback__)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
| res = hook_impl.function(*args)
| ^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/_pytest/python.py", line 162, in pytest_pyfunc_call
| result = testfunction(**testargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/tests/util/test_io.py", line 71, in test_reader_receive_line_reads_all_data
| @example(data=b"\r\n", rng=Random(0))
| ^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/core.py", line 1638, in wrapped_test
| raise the_error_hypothesis_found
| BaseExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pytest_trio/plugin.py", line 195, in _fixture_manager
| yield nursery_fixture
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/pytest_trio/plugin.py", line 250, in run
| await self._func(**resolved_kwargs)
| File "/home/jstanek/personal/project/shoji/tests/util/test_io.py", line 77, in test_reader_receive_line_reads_all_data
| recv_lines = [line async for line in reader]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/src/shoji/util/io.py", line 88, in __anext__
| return await self.receive_line()
| ^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/src/shoji/util/io.py", line 66, in receive_line
| if chunk := await self.stream.receive_some(max_len):
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/tests/util/test_io.py", line 36, in receive_some
| return next(self.source_iterator, b"")
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/src/shoji/testing/helper.py", line 45, in random_chunks
| end = start + rng.randint(min(min_size, remains), remains)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "<string>", line 5, in randint
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/strategies/_internal/random.py", line 132, in implementation
| result = self._hypothesis_do_random(name, kwargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/strategies/_internal/random.py", line 273, in _hypothesis_do_random
| result = self.__data.draw_integer(kwargs["a"], kwargs["b"])
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2093, in draw_integer
| value = self.provider.draw_integer(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 1461, in draw_integer
| return self._draw_bounded_integer(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 1769, in _draw_bounded_integer
| probe = self._cd.draw_bits(
| ^^^^^^^^^^^^^^^^^^^
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2574, in draw_bits
| self.__check_capacity(n_bytes)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2616, in __check_capacity
| self.mark_overrun()
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2641, in mark_overrun
| self.conclude_test(Status.OVERRUN)
| File "/home/jstanek/personal/project/shoji/.nox/test-3-12/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2628, in conclude_test
| raise StopTest(self.testcounter)
| hypothesis.errors.StopTest: 1366
+------------------------------------
-------------------------------------------------------- Hypothesis ---------------------------------------------------------
You can add @seed(282959970759348657142277025956285860727) to this test or run pytest with --hypothesis-seed=282959970759348657142277025956285860727 to reproduce this failure.
================================================== short test summary info ==================================================
FAILED tests/util/test_io.py::test_reader_receive_line_reads_all_data - BaseExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================== 1 failed, 10 passed in 2.58s ================================================
nox > Command python -m pytest --exitfirst failed with exit code 1
reproduces with:
@given(st.randoms())
def f(rng):
rng.randint(0, 1 << 25_000)
for i in range(10):
print(i)
f()
this is potentially not ir-related, I haven't tried to bisect yet.
https://github.com/HypothesisWorks/hypothesis/blob/218d8a17dbad4dea84b7bb950a9df7b93425301e/hypothesis-python/src/hypothesis/strategies/_internal/random.py#L272-L273
I think we get the StopTest here and somehow fail to turn it into an Overrun error.
I think I was likely on an outdated master when I claimed I had a reproducer, because I can't reproduce on latest hypothesis anymore and my reproducer just traces back to the fix in #3991. And yet, the traceback in OP indicates this is a distinct issue (ie not caused by generate_novel_prefix).
@khardix are you able to share the offending strategy here? I can't immediately reproduce with synthetic overruns.
@tybug Let me share the project so you have the complete picture, and then I'll point out the things I think are relevant.
https://git.sr.ht/~khardix/shoji, the offending tests live in tests/test__io.py. Setting HYPOTHESIS_RANDOM_BROKEN to False should trigger the bug.
Versions I'm able to trigger the bug on: hypothesis-6.108.10, shoji-0.3.1 (or current master branch: 487a74c7635650351cd3701f2a02170b5cf1dc31).
The random strategy is just randoms(use_true_random=False). However, it is used in conjuction with a composite strategy that generates a text-like byte string, which is then chopped to random chunks by the random strategy.
The composite strategy:
@stg.composite
def textlike_data(
draw: stg.DrawFn,
line_terminator: str = "\n",
min_line_len: int = 0,
max_line_len: int | None = _io.MAX_LINE_LEN,
) -> bytes:
"""Generate binary data that look like UTF-8 text.
Generates the data as a list of lines,
which makes it easier to reason about whether there is *any* line terminator preset.
"""
alphabet = stg.characters(codec="utf-8", exclude_characters=set(line_terminator))
line = stg.text(alphabet, min_size=min_line_len, max_size=max_line_len)
text = stg.lists(line)
final_terminator = stg.one_of(stg.just(""), stg.just(line_terminator))
return bytes(
line_terminator.join(draw(text)) + draw(final_terminator), encoding="utf-8"
)
The textlike_data and randoms examples are then fed into a generator that simulates random chunks of data coming from a network:
def random_chunks(
data: bytes, min_size: int = 0, *, rng: Random = RNG
) -> abc.Iterator[bytes]:
"""Split data into random-sized chunks.
Arguments:
data: The data to split.
min_size: Minimal size of a chunk.
..note:: The last chunk may be smaller than `min_size`, even empty!
Keyword arguments:
rng: Random Number Generator for determining the chunk sizes.
Yields:
Chunks of `data`.
Raises:
ValueError: When `min_size` is negative.
"""
if min_size < 0:
raise ValueError(f"Negative minimal size: {min_size}")
if len(data) < min_size:
yield data
return
start_idx = 0
while start_idx < len(data):
remaining_len = len(data) - start_idx
end_idx = start_idx + rng.randint(min(min_size, remaining_len), remaining_len)
yield data[start_idx:end_idx]
start_idx = end_idx
I suspect this is a bad interaction between hypothesis and pytest-trio:
@given(stg.data())
async def test_bufreciever_receive_line_reads_all_data(data):
# force an overrun
for _ in range(5):
data.draw(stg.integers(0, 1 << 25_000))
traceback
============================================================================ test session starts =============================================================================
platform darwin -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0
rootdir: /Users/tybug/Desktop/shoji
configfile: pyproject.toml
plugins: hypothesis-6.108.10, trio-0.8.0, anyio-4.4.0
collected 21 items / 20 deselected / 1 selected
tests/test__io.py F [100%]
================================================================================== FAILURES ==================================================================================
________________________________________________________________ test_bufreciever_receive_line_reads_all_data ________________________________________________________________
+ Exception Group Traceback (most recent call last):
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/runner.py", line 341, in from_call
| result: TResult | None = func()
| ^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/runner.py", line 242, in <lambda>
| lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
| return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
| return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 182, in _multicall
| return outcome.get_result()
| ^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_result.py", line 100, in get_result
| raise exc.with_traceback(exc.__traceback__)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
| yield from thread_exception_runtest_hook()
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
| yield
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
| yield from unraisable_exception_runtest_hook()
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
| yield
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/logging.py", line 848, in pytest_runtest_call
| yield from self._runtest_for(item, "call")
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/logging.py", line 831, in _runtest_for
| yield
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/capture.py", line 879, in pytest_runtest_call
| return (yield)
| ^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 167, in _multicall
| teardown.throw(outcome._exception)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
| return (yield)
| ^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
| res = hook_impl.function(*args)
| ^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
| item.runtest()
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/python.py", line 1627, in runtest
| self.ihook.pytest_pyfunc_call(pyfuncitem=self)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
| return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
| return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
| raise exception.with_traceback(exception.__traceback__)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
| res = hook_impl.function(*args)
| ^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
| result = testfunction(**testargs)
| ^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/tests/test__io.py", line 67, in test_bufreciever_receive_line_reads_all_data
| async def test_bufreciever_receive_line_reads_all_data(data):
| ^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/core.py", line 1699, in wrapped_test
| raise the_error_hypothesis_found
| BaseExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pytest_trio/plugin.py", line 195, in _fixture_manager
| yield nursery_fixture
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/pytest_trio/plugin.py", line 250, in run
| await self._func(**resolved_kwargs)
| File "/Users/tybug/Desktop/shoji/tests/test__io.py", line 69, in test_bufreciever_receive_line_reads_all_data
| data.draw(stg.integers(0, 1 << 25_000))
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/strategies/_internal/core.py", line 2115, in draw
| result = self.conjecture_data.draw(strategy, observe_as=f"generate:{desc}")
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2458, in draw
| return strategy.do_draw(self)
| ^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/strategies/_internal/lazy.py", line 167, in do_draw
| return data.draw(self.wrapped_strategy)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2452, in draw
| return strategy.do_draw(self)
| ^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/strategies/_internal/numbers.py", line 85, in do_draw
| return data.draw_integer(
| ^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2121, in draw_integer
| value = self.provider.draw_integer(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 1481, in draw_integer
| return self._draw_bounded_integer(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 1780, in _draw_bounded_integer
| probe = self._cd.draw_bits(
| ^^^^^^^^^^^^^^^^^^^
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2612, in draw_bits
| self.__check_capacity(n_bytes)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2654, in __check_capacity
| self.mark_overrun()
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2679, in mark_overrun
| self.conclude_test(Status.OVERRUN)
| File "/Users/tybug/Desktop/shoji/myenv/lib/python3.12/site-packages/hypothesis/internal/conjecture/data.py", line 2666, in conclude_test
| raise StopTest(self.testcounter)
| hypothesis.errors.StopTest: 1
+------------------------------------
--------------------------------------------------------------------------------- Hypothesis ---------------------------------------------------------------------------------
You can add @seed(102826716451383249480663158101555025900) to this test or run pytest with --hypothesis-seed=102826716451383249480663158101555025900 to reproduce this failure.
========================================================================== short test summary info ===========================================================================
FAILED tests/test__io.py::test_bufreciever_receive_line_reads_all_data - BaseExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
====================================================================== 1 failed, 20 deselected in 0.04s ======================================================================
pytest-trio 0.8.0, hypothesis 6-108.10.
pytest-trio may be swallowing StopTest and raising an exception group instead before StopTest can be excepted by __stoppable_test_function? which then bubbles up as we don't catch BaseExceptionGroup.
@Zac-HD maybe you have ideas here? 😅 I will likely be quite slow to identify the correct resolution with trio involved.
(thanks a lot for the reproducer @khardix!)
ooooohhh, a BaseExceptionGroup containing the StopTest would indeed explain this.
In the long term I'd really like Hypothesis to "just work" with (Base)ExceptionGroups; in the short term we might end up just catching and crashing on this as a more-interpretable improvement on the status quo. We do already depend on the exceptiongroup backport, but missing the except* syntax before py311 is kinda painful...
Closing this issue because https://github.com/HypothesisWorks/hypothesis/issues/4106 is a superset - we're working on it!
Thanks again for the report @khardix, and stay tuned for a fix 🙂