zed icon indicating copy to clipboard operation
zed copied to clipboard

count() in lateral query not resetting to zero

Open philrz opened this issue 2 years ago • 1 comments

tl;dr

@mccanne's statement upon seeing the repro below:

That looks like a bug! We're not resetting the streaming agg state at end-of-stream.

Details

Repro is with Zed commit 59b72d8. I bumped into this while working on a solution for the question posed in #4937.

Compare these two behaviors.

$ zq -version
Version: v1.11.1-27-g59b72d88

$ echo '["a","b","c","d","e","f","g"] ["h","i","j","k"]' | zq -z 'over this => (count())' -
7(uint64)
4(uint64)

$ echo '["a","b","c","d","e","f","g"] ["h","i","j","k"]' | zq -z 'over this => ({id:count()})' -
{id:1(uint64)}
{id:2(uint64)}
{id:3(uint64)}
{id:4(uint64)}
{id:5(uint64)}
{id:6(uint64)}
{id:7(uint64)}
{id:8(uint64)}
{id:9(uint64)}
{id:10(uint64)}
{id:11(uint64)}

The lateral subquery docs speak of how the subquery "is evaluated once per outer value" (i.e., per array in this case) so the top behavior made sense to me while the bottom one didn't. I'd expected the count() result to reset to zero when processing began on the second array value.

philrz avatar Dec 16 '23 19:12 philrz

Note to self: Update #4937 when this issue is fixed.

philrz avatar Dec 27 '23 20:12 philrz

Verified in Zed commit 711bf9f.

Repeating the original repro, the count() result nows resets to zero when processing begins on the second array value.

$ zq -version
Version: v1.15.0-34-g711bf9f4

$ echo '["a","b","c","d","e","f","g"] ["h","i","j","k"]' | zq -z 'over this => ({id:count()})' -
{id:1(uint64)}
{id:2(uint64)}
{id:3(uint64)}
{id:4(uint64)}
{id:5(uint64)}
{id:6(uint64)}
{id:7(uint64)}
{id:1(uint64)}
{id:2(uint64)}
{id:3(uint64)}
{id:4(uint64)}

Thanks @mattnibs!

philrz avatar Jun 07 '24 21:06 philrz