chronon icon indicating copy to clipboard operation
chronon copied to clipboard

[CI] Set a time limit for circle ci

Open cristianfr opened this issue 1 year ago • 0 comments

Summary

Setting a time limit of 40 minutes for circle CI as metadata calls can run into a 4hr+ loop of failure Example: https://app.circleci.com/pipelines/github/airbnb/chronon/3408/workflows/2fdc3dd8-904b-4d90-b062-d5879f4bd686/jobs/17449

From: https://support.circleci.com/hc/en-us/articles/11775183948827-How-to-run-a-job-in-the-background-on-CircleCI

A job that has background: true set will run while there are other jobs running but once they have finished the background job will also end, it will not run on its own if there are no other jobs.

Pick 40 minutes since both Spark 3.1.1 and 3.2 run in ~10-20minutes. Spark 2 can run for 40 minutes but rarely will provide more info than the other two together.

Why / Goal

Improve dev experience.

Test Plan

  • [ ] Added Unit Tests
  • [x] Covered by existing CI
  • [ ] Integration tested

Checklist

  • [ ] Documentation update

Reviewers

anyone.

cristianfr avatar Oct 27 '23 22:10 cristianfr