subgrounds
subgrounds copied to clipboard
Backfilling, checkpointing, improved pagination
Is your feature request related to a problem? Please describe. I want better tools when working with large subgrounds queries.
I often deal with queries that can take overnight, so more tools to help manage the lifecycle of those queries can help me ensure the success of those queries.
Describe the solution you'd like
- Improved pagination strategies that helps me recover when a graph node spits in error out late in the pagination cycle
- Being able to checkpoint or backfill queries so that when errors do occur, I'm able to recover w/o any data-loss
Describe alternatives you've considered
Using _iter
methods to help checkpoint data
- Difficult and not-well documented (also should be fixed)
- Generally would be easier to have
subgrounds
handle this
Additional context Opinionated so should result in multiple solutions
Implementation checklist
- [ ] Task 1