backend icon indicating copy to clipboard operation
backend copied to clipboard

AI-powered marking

Open xxdydx opened this issue 7 months ago • 5 comments

Description

Created a new feature to automate feedback generation of code submissions in Source Academy, with the help of LLMs. This will help save TAs some time in grading code submissions from students!

  • Added AI comment generation functionality in the backend.
  • Integrated OpenAI API for generating AI comments.
  • Implemented generate_ai_comments endpoint to fetch question details and generate AI-generated comments for submissions.
  • Added save_chosen_comments endpoint to save multiple chosen comments for a submission and question for logging purposes.
  • Added save_final_comment endpoint to save the final comment chosen for a submission for logging purposes.
  • Added new ai_comment_logs table to log various data points from inputs, original student's code, outputs generated by LLM, comments chosen, and final comment.
  • Added AIComments module to handle creation, retrieval, and updates for AI comments, including saving final and chosen comments.
  • Added AICodeAnalysisController to handle AI comment generation, saving final comments, and saving chosen comments.
  • Added test cases for generate_ai_comments, save_final_comment, and save_chosen_comments endpoints in AICodeAnalysisControllerTest.
  • Updated Swagger documentation for the new endpoints.
  • Added necessary migrations to update the database schema.
  • Added encryption and decryption logic for LLM API keys using AES-GCM.

Note: This may require changes to the DB diagram in README.md.

Type of Change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [x] This change requires a documentation update
  • [ ] Code quality improvements

Checklist

  • [x] I have tested this code
  • [ ] I have updated the documentation

xxdydx avatar Apr 06 '25 06:04 xxdydx

Coverage Status

coverage: 88.716% (-0.9%) from 89.621% when pulling d0a08df181cfe2ee294f3f618b8aceeddda8e581 on feat/add-AI-generated-comments-grading into 8736877eace8420f18a184c33a66e1c785dbcd1d on master.

coveralls avatar Apr 09 '25 05:04 coveralls

@sentry review

RichDom2185 avatar Oct 26 '25 14:10 RichDom2185

@sentry review

RichDom2185 avatar Oct 26 '25 14:10 RichDom2185

@sentry generate-test

RichDom2185 avatar Oct 31 '25 04:10 RichDom2185

Sentry has determined that unit tests are not necessary for this PR.

seer-by-sentry[bot] avatar Oct 31 '25 04:10 seer-by-sentry[bot]