specification icon indicating copy to clipboard operation
specification copied to clipboard

[WIP] feat: [#1096] While, Do-while loop implementation for fetching paginated data from an API

Open nyamathshaik opened this issue 8 months ago • 5 comments

Please specify parts of this PR update:

  • [x] Specification
  • [x] Schema
  • [x] Examples
  • [ ] Extensions
  • [x] Use Cases
  • [ ] Community
  • [ ] CTK
  • [ ] Other Discussion or Issue link: https://github.com/serverlessworkflow/specification/issues/1096

What this PR does: This PR introduces enhancements to support iterative logic within the do block using a while loop construct. Specifically, it adds support for:

A while condition to allow loop execution as long as the condition evaluates to true.

postConditionCheck: false to enable do-while loop semantics.

maxIterations to define an upper limit on loop executions, providing safety against infinite loops.

The changes are accompanied by an example paginated-fetch-example that demonstrates fetching paginated API data until .hasNextPage == true, accumulating results across iterations.

Additional information: This capability enables richer control flow for iterative data-fetching use cases in serverless workflows, reducing the need for complex custom code and improving the expressiveness of the DSL.

Sample document:

document:
  dsl: '1.0.0'
  namespace: test
  name: paginated-fetch-example
  version: '0.1.0'

do:
  - fetchPaginatedData:
      condition: .hasNextPage == true
      isDoWhile: true # Enables do-while behavior
      limit: 100
      do:
        - fetchPage:
            call: getPageData
            input:
              pageToken: .nextPageToken
            output: .fetchedData
        - accumulateData:
            run: mergeResults
            input:
              newData: .fetchedData
              existingData: .accumulatedResults
            output: .accumulatedResults

nyamathshaik avatar Apr 14 '25 10:04 nyamathshaik

Hi, @nyamathshaik and thanks for your work!

However, as per contribution guidelines, I have to ask you to please refrain from opening a PR, especially not in draft, before gaining approval from all maintainers, as your feature request might be rejected (even though, in this case, it probably won't be, but the implementation details still remain to be defined)

cdavernas avatar Apr 14 '25 10:04 cdavernas

I have a couple of remarks regarding your proposal, which I provided in the related issue.

cdavernas avatar Apr 14 '25 10:04 cdavernas

@cdavernas Thanks for letting me know! 😊 Could you please take a look at the issue #1096 and let me know if the proposal looks good to move forward?

Happy to incorporate any feedback or iterate further based on the maintainers' suggestions.

nyamathshaik avatar Apr 14 '25 10:04 nyamathshaik

@nyamathshaik I think this is a WIP, right? Since we haven't settled the conversation yet.

ricardozanini avatar Apr 14 '25 17:04 ricardozanini

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar May 31 '25 00:05 github-actions[bot]