asyncstdlib icon indicating copy to clipboard operation
asyncstdlib copied to clipboard

[Bug]: Inconsistent StopIteration Handling in asyncstdlib.accumulate

Open Atry opened this issue 7 months ago • 3 comments

What happened?

asyncstdlib.accumulate handles StopIteration exceptions inconsistently compared to itertools.accumulate. When a custom accumulator function raises StopIteration, itertools.accumulate correctly stops iteration, while asyncstdlib.accumulate converts it to a RuntimeError.

Expected Output

=== itertools.accumulate behavior ===
Successfully stopped, result: [0, 1]

=== asyncstdlib.accumulate behavior ===
Successfully stopped, result: [0, 1]

Actual Output

=== itertools.accumulate behavior ===
Successfully stopped, result: [0, 1]

=== asyncstdlib.accumulate behavior ===
Exception: RuntimeError: coroutine raised StopIteration

Minimal Reproducible Example

import asyncstdlib
import itertools

# Accumulator function: raises StopIteration when accumulated value reaches 3
def accumulator_func(x, y):
    if x + y >= 3:
        raise StopIteration("Stopping at 3")
    return x + y

async def async_numbers():
    """Async generator: yields 0, 1, 2, 3, 4"""
    for i in range(5):
        yield i

def sync_numbers():
    """Sync generator: yields 0, 1, 2, 3, 4"""
    for i in range(5):
        yield i

# Test itertools.accumulate (expected behavior)
print("=== itertools.accumulate behavior ===")
try:
    result = list(itertools.accumulate(sync_numbers(), accumulator_func))
    print(f"Successfully stopped, result: {result}")
except Exception as e:
    print(f"Exception: {type(e).__name__}: {e}")

print("\n=== asyncstdlib.accumulate behavior ===")
try:
    result = []
    async for item in asyncstdlib.accumulate(async_numbers(), accumulator_func):
        result.append(item)
    print(f"Successfully stopped, result: {result}")
except Exception as e:
    print(f"Exception: {type(e).__name__}: {e}")

Request Assignment [Optional]

  • [x] I already understand the cause and want to submit a bugfix.

Atry avatar Aug 07 '25 21:08 Atry

Thanks for the report. This is tricky.

Raising RuntimeError is the correct behaviour as per PEP 479 (and in fact happens automatically); the regular accumulator function should not raise Stop[Async]Iteration since it's not an (async) iterator, and the RuntimeError reports this misuse.

This seems rather inconsistently handled in the stdlib (because most iterators are hand-built, not generators). For reference, the closely related functools.reduce lets the StopIteration bubble up directly. My midnight hunch is that both of these standard library functions are bugged.

I have to ponder whether consistency with the sync implementation or with other such iterators is preferable.

maxfischer2781 avatar Aug 07 '25 21:08 maxfischer2781

PEP 479 is about generaor, not about how an iterator's __next__ behaves. Allowing a callback function passed to accumulate or map to early stop the iteration is a feature, no matter whther accumulate or map is implemented in a generator or a manual iterator. Even when it's implemented in generators, it is possible to catch the Stop[Async]Iteration raised by the callback and convert it into an early return.

Atry avatar Aug 18 '25 16:08 Atry

PEP 479 is about generaor, not about how an iterator's __next__ behaves. […]

I am aware. The rationale applies equally to iterator's __next__; it's just not possible to enforce the same way as for generators.

Allowing a callback function passed to accumulate […] to early stop the iteration is a feature […]

I am not convinced of that. The accumulate docs make no mention of it and the Python "equivalent" does not handle this case even though it handles StopIteration for another case. Handling StopIteration the way it does now seems purely incidental side-effect of the implementation.

Note both map and accumulate handle this incorrectly (Py 3.12.10): A StopIteration is propagated but does not actually end the iterator.

>>> m = map(stop, range(6))
... for i in range(6):
...     print(i, next(m,  None))
0 0
1 1
2 2
3 None
4 4
5 5

This violates the data model: (emphasis mine)

Once an iterator’s next() method raises StopIteration, it must continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken.

maxfischer2781 avatar Aug 18 '25 17:08 maxfischer2781