GSoC-Ideas icon indicating copy to clipboard operation
GSoC-Ideas copied to clipboard

Published 20 hours ago • verified publisher icon Cloud-CV

Seamless User Experience & Leaderboard Porting

Open gchhablani opened this issue 1 year ago • 4 comments

Project Title: Seamless User Experience & Leaderboard Porting

Description: This project aims to enhance the overall user experience on EvalAI by introducing user-centric features, improving existing functionalities, and optimizing integrations with external platforms.

Two key features that we aim to implement in this project will be:

  • Allowing custom requests on code-upload and static code-upload challenges from within the evaluation workers running on EKS platform.
  • Making enhancements to the EvalAI in an efficient way to to migrate existing leaderboards for external challenges from various sources (maybe a tabular format) to EvalAI as the hosts migrate their challenges to EvalAI.

These features, coupled with additional deliverables below, will improve the overall experience of EvalAI users and add several utilities for improving challenge participation experience.

Deliverable:

  • Code-Upload Challenges Improvements:
    • [ ] Allow custom requests for EKS challenges to external sites like HuggingFace and OpenAI for model downloads.
    • [ ] Address sporadic pod log population on AWS and address the existing issue of environment logs not getting populated on our backend from CloudWatch.
  • Leaderboard Migration:
    • [ ] Provide a solution for transferring existing challenge leaderboards from external sources to EvalAI.
    • [ ] This involves adding an ability to take a CSV (or any other format) of participant teams, leaderboard metrics, submission timestamp, challenge name and convert it to an “archived” entry on the public leaderboard to see.
  • User-centric Features:
    • [ ] Implement a customizable participant details download option on the analytics dashboard.
    • [ ] Improve error handling and display for email verification issues.
    • [ ] Embed the forum on the challenge page for easy communication between hosts and participants.
  • Additional User Experience Improvements:
    • [ ] Fix the search and filter feature PR and perform robust testing on the same.
    • [ ] Automate challenge category/forum creation on EvalAI forum during challenge approval.

Mentor: Rahul Singh, @gautamjajoo , @gchhablani @RishabhJain2018

Skills: SQL, Django, AngularJS, AWS

Skill Level: Medium

Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSoC-2024).

Important Links:

gchhablani avatar Feb 09 '24 19:02 gchhablani