dagster
dagster copied to clipboard
Change the default implementation to calculate step stats more efficiently
Summary & Motivation
By default, we calculate step stats by fetching all of the events and filtering in-memory to construct the step stat summary.
This PR changes the event fetch to pass in an event type filter.
This has the biggest impact for runs that have a lot of unstructured events and that do not directly inherit from the SqlEventLogStorage (e.g. some Cloud storage implementations)
How I Tested These Changes
BK