mergestat-lite
mergestat-lite copied to clipboard
feat(actions): First iteration of GITHUB_ACTIONS sync
With this pull request we open our first iteration over the new sync type GITHUB_ACTIONS this one is a complex one because itself divides in three main categories related in order workflows-> workflow runs -> workflow jobs -> workflow logs Also we don't use the graphql client neither the mergestat-light dependency for this sync .Mainly because we are moving away from mergestat-light dependency and the graphql api from Github does not provided the information we require for this sync .
*NOTE * There are a lot of transient errors from GitHub API that we are not handling in this PR , those errors will be handled in #408 .
For this specific problem we choosed this algorithm to manage the iteration of getting all github actions
`
-
WorkflowIterator(owner,repo,repoID)
- Pull a pages of Workflows from the API
- For each workflows_page
- HandleWorkflowsUpsert([]workflows ,repoID)
- WorkflowrunsIterator(owner,repo,[]workflows)
- For each workflows_page
- Pull a pages of Workflows from the API
-
WorkflowrunsIterator(owner,repo,[]workflows,repoID)
- Pull a page of Workflow runs from the API
- For each workflowRuns_Page
- HandleWorkflowRunsUpsert([]workflowRuns,repoID)
- WorkflowjobsIterator(owner,repo,[]workflowRuns)
- For each workflowRuns_Page
- Pull a page of Workflow runs from the API
-
WorkflowjobsIterator([]runs)
- Pull a pages of Workflow jobs from the API
- For each []jobs_page
- Per job we pull a log getWorkflorJobLog(owner,repo,jobID)
- HandleUpsertWorkflowJob (job, log,repoID) `
- For each []jobs_page
and a picture :) resolves #366
#367 #368 - Pull a pages of Workflow jobs from the API
