burr
burr copied to clipboard
Configurable Persistence Frequency
Is your feature request related to a problem? Please describe. Currently we persist after every node. We should allow this to be configurable. This saves you from an over-saving state, which can cause slowness/state bloat.
Describe the solution you'd like A few ideas:
- Add a filter in the
with_persisterargument -- probably node names (exclude/include) - Add in the Action itself (E.G. have an action say if it should be persisted/tracked/transient)
- Integrate with tags (feature we don't have yet) -- have a
tags=['no_persist']or something -- see https://github.com/DAGWorks-Inc/burr/issues/468
So really, do we want this at the action or application level? My instinct is that we can have both but the action level is more valuable, need to think it through though...
Describe alternatives you've considered See above + https://github.com/DAGWorks-Inc/burr/issues/468.