[#4670]Add bi-directional sync feature
This PR in tackling issue #4760, enables syncing Challenge and Challenge Phase updates made via the EvalAI UI to the configured GitHub repository/branch. When a supported field changes in the UI, a corresponding commit is made to the repo to keep configuration in sync.
EvalAI-Starters side : https://github.com/Cloud-CV/EvalAI-Starters/pull/129
How it works
- Post-save hooks in
apps/challenges/models.py:challenge_details_syncforChallengechallenge_phase_details_syncforChallengePhase
- Determines the changed field from
update_fieldsor infers it from the request payload keys to make a minimal, single-field commit. - Calls
challenges.github_utilssync helpers to apply changes on GitHub.
Safeguards
- Skips sync for:
- Bot-triggered saves (prevents recursion)
- GitHub-sourced changes (prevents loops)
- Repeated syncs within the same request (per-request dedupe)
- Requires
github_repository,github_branch, andgithub_tokento be set onChallenge.
Configuration
- Set on each
Challenge:github_repository(e.g., org/repo)github_branch(defaults handled in code)github_token(PAT with repo access)
Migrations
0113_add_github_branch_field_and_unique_constraint.pynow:- Adds
github_branchif missing (SQL) - Records the field in Django state (no-op DB)
- Adds a partial unique index on (
github_repository,github_branch) when both are non-empty
- Adds
- Removed redundant
0116_challenge_github_branch.pyso there’s a single source of truth.
Django signals
- Receivers:
challenge_details_synconChallenge(post_save)challenge_phase_details_synconChallengePhase(post_save)
- Behavior:
- Trigger only on updates when
github_repositoryandgithub_tokenare set. - Determine the changed field via
update_fieldsor inferred request payload keys. - Commit a minimal, single-field change to GitHub through
challenges.github_utils.
- Trigger only on updates when
- Guards:
- Skip bot-triggered saves and GitHub-sourced changes to prevent recursion/loops.
- Per-request dedupe avoids duplicate commits in the same request.
GitHubSyncMiddleware
- Purpose: Resets per-request sync context and captures payload keys for changed-field inference.
- How:
- On
POST/PUT/PATCH, extracts JSON or form keys into a thread-local store. - Works with the signal receivers to identify the precise field changed.
- Prevents re-entrant syncs within the same request.
- On
Dependencies
- requests: Added/confirmed in
requirements/common.txtto support GitHub API calls used byapps/challenges/github_interface.pyand related utilities.
Here is a video description of the feature : https://github.com/user-attachments/assets/f9701bf2-3729-4998-8ec3-9d060362965e
Codecov Report
:x: Patch coverage is 56.81234% with 168 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 90.09%. Comparing base (3489634) to head (4a05fa5).
:warning: Report is 2 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #4765 +/- ##
==========================================
- Coverage 91.90% 90.09% -1.82%
==========================================
Files 85 88 +3
Lines 7142 7531 +389
==========================================
+ Hits 6564 6785 +221
- Misses 578 746 +168
| Flag | Coverage Δ | |
|---|---|---|
| backend | 92.70% <56.81%> (-3.96%) |
:arrow_down: |
| frontend | 87.25% <ø> (ø) |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Components | Coverage Δ | |
|---|---|---|
| Accounts & Authentication | 100.00% <ø> (ø) |
|
| Challenges Management | 89.72% <57.29%> (-5.63%) |
:arrow_down: |
| Job Processing | 98.19% <ø> (ø) |
|
| Participants & Teams | 100.00% <ø> (ø) |
|
| Challenge Hosts | 100.00% <ø> (ø) |
|
| Analytics | 100.00% <ø> (ø) |
|
| Web Interface | 100.00% <ø> (ø) |
|
| Frontend (Gulp) | 87.25% <ø> (ø) |
|
| All Models | 94.38% <78.26%> (-3.21%) |
:arrow_down: |
| All Views | 100.00% <ø> (ø) |
|
| All Serializers | 98.50% <66.66%> (-0.32%) |
:arrow_down: |
| Utility Functions | 96.51% <20.00%> (-0.29%) |
:arrow_down: |
| Core Configuration | 82.35% <ø> (ø) |
| Files with missing lines | Coverage Δ | |
|---|---|---|
| apps/challenges/urls.py | 100.00% <ø> (ø) |
|
| apps/challenges/views.py | 100.00% <ø> (ø) |
|
| apps/challenges/serializers.py | 96.21% <66.66%> (-0.77%) |
:arrow_down: |
| apps/base/utils.py | 95.74% <20.00%> (-2.79%) |
:arrow_down: |
| apps/challenges/github_sync_config.py | 0.00% <0.00%> (ø) |
|
| apps/challenges/github_utils.py | 64.91% <64.91%> (ø) |
|
| apps/challenges/models.py | 91.33% <78.26%> (-4.49%) |
:arrow_down: |
| apps/challenges/github_interface.py | 44.27% <44.27%> (ø) |
| Files with missing lines | Coverage Δ | |
|---|---|---|
| apps/challenges/urls.py | 100.00% <ø> (ø) |
|
| apps/challenges/views.py | 100.00% <ø> (ø) |
|
| apps/challenges/serializers.py | 96.21% <66.66%> (-0.77%) |
:arrow_down: |
| apps/base/utils.py | 95.74% <20.00%> (-2.79%) |
:arrow_down: |
| apps/challenges/github_sync_config.py | 0.00% <0.00%> (ø) |
|
| apps/challenges/github_utils.py | 64.91% <64.91%> (ø) |
|
| apps/challenges/models.py | 91.33% <78.26%> (-4.49%) |
:arrow_down: |
| apps/challenges/github_interface.py | 44.27% <44.27%> (ø) |
Continue to review full report in Codecov by Sentry.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 3489634...4a05fa5. Read the comment docs.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.