mypy icon indicating copy to clipboard operation
mypy copied to clipboard

[mypyc] feat: constant folding for DictExpr [1/1]

Open BobTheBuidler opened this issue 4 months ago • 3 comments

It should be possible to "constant fold" a LOC such as:

requests.get(some_url, headers={"k0": "v0", "k1": "v1"})

We can't simply create a dictionary constant since dicts are mutable and we don't know what's going to happen in the called function, but we can hold a template dict in memory and PyDict_Copy it when it's time for use. This skips the hashing and any resizing during construction, and simply takes the hash table from the existing dict instead, which will be (possibly much) quicker than constructing from scratch each time.

This PR does that.

BobTheBuidler avatar Sep 17 '25 06:09 BobTheBuidler

Nothing specific I can readily share but based on my own use case I see it being useful in any code that's making requests and sending consistent or quasi-consistent headers along with them, or code that creates a dict by starting from some base dict expr and modifying it with conditional logic below

For me this is most meaningful for a high thruput internal API

On Tue, Oct 14, 2025 at 8:29 AM Jukka Lehtosalo @.***> wrote:

@.**** commented on this pull request.

I verified that this makes a simple micro-benchmark about 20% faster. However, this is quite a big and complex PR, and I'm not sure if the use case is common enough to support the "complexity budget". Can you show some real-world use cases where this would help?

— Reply to this email directly, view it on GitHub https://github.com/python/mypy/pull/19864#pullrequestreview-3335353257, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ3HIHR7CQ7Q2LVHXJQD44L3XTUCBAVCNFSM6AAAAACGXC4PYGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTGMZVGM2TGMRVG4 . You are receiving this because you authored the thread.Message ID: @.***>

BobTheBuidler avatar Oct 14 '25 12:10 BobTheBuidler

Also helps for functions that need their own little lookup tables that aren't worthy of being a module level constant. I have a bunch of these little lookup tables in one repo I maintain

example:


def check_owner(x: LicensePlate) -> bool:
    owner = {"honda": "tom", "ford": "tim", "vw": "tam"}[x.car]
    return is_good(owner)
    

edit

On Tue, Oct 14, 2025 at 8:33 AM Bob Buidler @.***> wrote:

Nothing specific I can readily share but based on my own use case I see it being useful in any code that's making requests and sending consistent or quasi-consistent headers along with them, or code that creates a dict by starting from some base dict expr and modifying it with conditional logic below

On Tue, Oct 14, 2025 at 8:29 AM Jukka Lehtosalo @.***> wrote:

@.**** commented on this pull request.

I verified that this makes a simple micro-benchmark about 20% faster. However, this is quite a big and complex PR, and I'm not sure if the use case is common enough to support the "complexity budget". Can you show some real-world use cases where this would help?

— Reply to this email directly, view it on GitHub https://github.com/python/mypy/pull/19864#pullrequestreview-3335353257, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ3HIHR7CQ7Q2LVHXJQD44L3XTUCBAVCNFSM6AAAAACGXC4PYGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTGMZVGM2TGMRVG4 . You are receiving this because you authored the thread.Message ID: @.***>

BobTheBuidler avatar Oct 14 '25 13:10 BobTheBuidler

@JukkaL here is an example of a file in the wild where this PR would make a meaningful difference: https://github.com/BobTheBuidler/py-solc-ast/blob/master/solcast/dependencies.py

BobTheBuidler avatar Dec 08 '25 06:12 BobTheBuidler