nbgrader
nbgrader copied to clipboard
jupyter
Autograde cell with input or print statemes
- Is it possible to autograde a cell that contains an input statement?
- Is it possible to autograde a cell that contains a print statement?
Operating system
OS X 10.14.6
nbgrader --version
Python version 3.7.3 (default, Mar 27 2019, 16:54:48) [Clang 4.0.1 (tags/RELEASE_401/final)] nbgrader version 0.6.0
jupyterhub --version (if used with JupyterHub)
jupyter notebook --version
6.0.0
Expected behavior
Actual behavior
Steps to reproduce the behavior
I'm not sure this is an nbgrader issue, but I'm also interested to know whether there is a possibility to autograde cells containing an input() statement. Regarding print() statements, perhaps the nbgrader documentation on autograding could be helpful.
I'm also interested in how to handle input statements, since they seem to prevent the rest of the cell from being run during validation, which means the notebooks can fail to validate even when all individual test cells pass when executed in order. I find this causes problems for beginner level assignments where input is used to demonstrate how variables work before concepts like iteration and functions have been introduced.
One workaround is to put a cell at the top of the notebook that overrides the input function so it simulates the correct output without requiring user interaction. As long as students do not run this cell manually, they will get the expected functionality while doing the assignment and the notebook will still pass validation when it should.
The obvious problem with this approach is that it clutters up the assignment with code that may be confusing to the students. If there is some way to detect that we are in validation-mode, a nicer solution would be to put the override as a conditional in the autograder tests, where it is less likely to distract students.
The obvious problem with this approach is that it clutters up the assignment with code that may be confusing to the students. If there is some way to detect that we are in validation-mode, a nicer solution would be to put the override as a conditional in the autograder tests, where it is less likely to distract students.
This may be relevant to your interests: NBGRADER_VALIDATING
environment variable set when validating, https://github.com/jupyter/nbgrader/pull/1061.
Someone should add this to the docs... looks at self.
This may not be an ideal solution, but this seems to be working for me so far. Sorry for the formatting issues ... Usage: Student would have defined a function called function_to_test() in the autograded answer cell, ie:
def function_to_test(): ____phrase = input("Enter your phrase >> ") ____print( phrase.replace(" ","...") ) function_to_test()
In the autograder tests cell, put:
from unittest.mock import patch old_input = input inputs = ("one two three", "alpha", "This is a great test.") outputs = ("one...two...three", "alpha", "This...is...a...great...test.") try: ____for i in range(len(inputs)): ________def input(p=''): ____________return inputs[i] ________with patch('main.print') as mock_print: ____________function_to_test() ________mock_print.assert_called_once_with(outputs[i]) except Exception as e: ____print(e) ____input = old_input ____raise Exception("Test[s] failed") input = old_input print("All tests pass!")
