hail icon indicating copy to clipboard operation
hail copied to clipboard

[query] map in python should allow requiring memory management per row

Open patrick-schultz opened this issue 4 months ago • 0 comments

If a user writes a map over a local array or stream value, which they know is large, they should be able to force scratch memory to be cleared after each row. This can be an underscored parameter for expert use only.

We would need to add a requires_memory_management_per_row flag to the StreamMap node, and expose it in user facing methods which generate a StreamMap like ArrayExpression.map. Then the rule in EmitStream would need to combine this with the child stream's requiresMemoryManagementPerRow.

patrick-schultz avatar Feb 29 '24 16:02 patrick-schultz