Add source code context to assertion failures in terminal environment
This PR enhances the terminal environment to provide source code context for algorithmic errors (assertion failures), bringing them to parity with runtime and syntax errors which already show helpful context.
Problem
Previously, when an assertion like assert_equal(call("add", 1, 2), 5) failed, students would only see:
✗ Failed Instructor Test
Student code failed instructor test.
I ran the code:
add(1, 2)
The value of the result was:
3
But I expected the result to be equal to:
5
While runtime errors already provided much better context:
✗ Type Error
I ran your code.
A TypeError occurred:
Unsupported operand type(s) for +: 'int' and 'str'
Line 3 of file runtime.py
json.loads("1") + ""
^^^^^^^^^^^^^^^^^^^^
Solution
This PR adds source code context to assertion failures by:
-
Creating source context utilities (
pedal/utilities/source_context.py) that can locate function definitions in the student's AST and extract relevant source code lines. -
Enhancing RuntimeAssertionFeedback to detect function calls in assertion contexts and automatically include the function definition location and source code.
-
Maintaining backward compatibility - assertions without function calls work exactly as before.
Result
Now when the same assertion fails, students see:
✗ Failed Instructor Test
Student code failed instructor test.
I ran the code:
add(1, 2)
The value of the result was:
3
But I expected the result to be equal to:
5
In your function add on line 1:
def add(x: int, y: int) -> int:
This provides students with immediate context about WHERE in their code the problem is occurring, making debugging significantly easier.
Implementation Details
- Added
find_function_definition()to locate functions in student AST by name - Added
get_source_line_context()to extract source code around a given location - Modified
RuntimeAssertionFeedbackmessage template to include{source_context}field - Enhanced assertion feedback to detect function calls and provide location context
- Handles edge cases gracefully (missing functions, multiple contexts, complex signatures)
The enhancement is minimal and surgical - it only activates when an assertion fails and a function call is detected in the context, with no performance impact on passing tests.
Fixes #118.
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.