mypy icon indicating copy to clipboard operation
mypy copied to clipboard

[mypyc] feat: ForFilter generator helper for `builtins.filter`

Open BobTheBuidler opened this issue 5 months ago • 8 comments

This PR adds a ForFilter helper class for for loops over builtins.filter which allows mypyc to maintain control of the function calling, so it can call native functions and use primitive ops, and to optimize filter's boolean check based on func's return value type

BobTheBuidler avatar Aug 12 '25 02:08 BobTheBuidler

Just realized I can extend this class to make it work for both builtins.filter and itertools.filterfalse with only 4 more LOC

I wouldn't think to make a primitive for iteration solely over itertools.filterfalse, but considering it would share the same object and only require one more boolean attribute, I think it makes sense to support here. What do you guys think?

BobTheBuidler avatar Aug 12 '25 13:08 BobTheBuidler

I went ahead and implemented the above on a new branch here, we can merge it into this branch if we decide its worth special casing. I'd say it probably is, considering the code change is quite minimal change once we already have ForFilter defined.

BobTheBuidler avatar Aug 13 '25 06:08 BobTheBuidler

I don't yet think the failing docs workflow is related to this PR, given that I only added a single bullet point to the native_operations.rst, though can't say for sure

BobTheBuidler avatar Aug 13 '25 14:08 BobTheBuidler

I wouldn't think to make a primitive for iteration solely over itertools.filterfalse, but considering it would share the same object and only require one more boolean attribute, I think it makes sense to support here. What do you guys think?

filterfalse seems to be used quite rarely (on the order of 100x less often than filter, based on a large codebase I have access to), so I think it's not worth supporting it at this time. My general guidance is to avoid itertools in any performance-sensitive code, so not having any support any of these makes it easier to teach users what to avoid. Otherwise users would have to remember which functions in itertools have optimized support.

JukkaL avatar Aug 15 '25 13:08 JukkaL

Thanks for the feedbacks. I have a few busy days ahead of me so I'll have to table these for now, but can try to get them all fixed up next week.

BobTheBuidler avatar Aug 15 '25 15:08 BobTheBuidler

Some tests are failing. Do yo need help with the failures?

JukkaL avatar Sep 08 '25 13:09 JukkaL

Yeah I could definitely use some help with this one, I'm not sure which recent change(s) caused this err nor what to do about it

Is this just another typeshed thing?

I also have one open question above from one of your review comments. I just tagged you in it to notify.

BobTheBuidler avatar Sep 08 '25 13:09 BobTheBuidler

Okay I've committed that change, once the tests finish the remaining IR errors should be related to the builder.accept comments above and will go away once we resolve that

BobTheBuidler avatar Sep 08 '25 14:09 BobTheBuidler