EvalAI-Starters icon indicating copy to clipboard operation
EvalAI-Starters copied to clipboard

Add CI Pipeline for Validating `challenge_config.yml`

Open Eeshu-Yadav opened this issue 9 months ago • 7 comments

Description

This PR adds a GitHub Actions workflow to automatically validate the challenge_config.yml file whenever a PR is opened or merged into the master branch. The validation ensures that the configuration file adheres to the required schema and that all referenced files (e.g., templates, annotations) exist.

Changes

  • Added .github/workflows/ci.yml:
    • Runs on PRs and pushes to master
    • Validates challenge_config.yml

Related Issues

  • Fixes #98

Checklist

  • [x] Added GitHub Actions workflow

Eeshu-Yadav avatar Mar 18 '25 13:03 Eeshu-Yadav

Try re-using this script if possible: https://github.com/Cloud-CV/EvalAI-Starters/blob/master/github/challenge_processing_script.py

RishabhJain2018 avatar Mar 18 '25 16:03 RishabhJain2018

@RishabhJain2018 even after putting the correct credentials , it is failing and showing logs in github actions like this : https://github.com/Eeshu-Yadav/EvalAI-Starters/issues/3 any suggestions ?

Eeshu-Yadav avatar Mar 20 '25 05:03 Eeshu-Yadav

Hey @Eeshu-Yadav , You need to only use this API: https://github.com/Eeshu-Yadav/EvalAI-Starters/blob/b3898846b718e00c938981995ea4b6b74516ad20/github/config.py#L7 and not use the other API for creating the challenge. We just need to validate if the challenge config is valid even when any PR is merged in the repository.

RishabhJain2018 avatar Mar 20 '25 05:03 RishabhJain2018

Hey @Eeshu-Yadav , Instead of changing the current python script, I would suggest you to add a new script. The current script is already being utilized in a feature.

RishabhJain2018 avatar Mar 20 '25 18:03 RishabhJain2018

okk @RishabhJain2018 sir , will work on that and raise a pr soon , Thanks

Eeshu-Yadav avatar Mar 22 '25 05:03 Eeshu-Yadav

Hey @Eeshu-Yadav , Any updates on this?

RishabhJain2018 avatar Mar 25 '25 07:03 RishabhJain2018

@RishabhJain2018 extremely sorry for the delay , yesterday my exams finished , will raise a pr in the evening , working on that

Eeshu-Yadav avatar Mar 28 '25 09:03 Eeshu-Yadav

No worries. Thanks for the update.

RishabhJain2018 avatar Mar 28 '25 14:03 RishabhJain2018