Placing solution delimiters around a subset of one line of code

[rgerkin]

My students have so little programming experience that I would like to have code blocks that look sort of like this:

x_plus_one = x + ...

where their job is to replace ... with 1. I'm not sure how to do that since there is some expectation that the solution delimiters be able to stand on their own, i.e. as comments on a separate line. The only way around this that I can think of is really crazy, something like:

class BeginSolution:
  def __init__(self, x):
    self.x = x
  def end_solution(self):
    return self.x

c = get_config()
c.ClearSolutions.begin_solution_delimeter = "BeginSolution("
c.ClearSolutions.end_solution_delimeter = ").end_solution()"
c.ClearSolutions.code_stub = {
    "python": "...",
}

x_plus_one = x + BeginSolution(1).end_solution()

which should successfully validate, replace the code stubs correctly, and allow me to write a test cell asserting that x_plus_one == x+1. But it's also a ridiculous hack. Is there any more elegant way to combine the hint or fixed part of the solution, and the student-driven part of the solution, into a single line?

[jhamrick]

Hmm, good question. (Have you tested using the config above? I'd be surprised actually if that works, since you're right that the assumption is that the solution is always at least one full line)

Unfortunately, I don't think there's an elegant way to do what you want at the moment, but I think it could be a nice feature to have a different set of solution delimiters that can deal with single lines, just like there's different types of comments in other programming languages which are either single line or multiline.

[rgerkin]

I haven't tested it, but I can't imagine actually using something like that in production. I think I'll wait until there is a more robust option for one-liners.

Also, I deployed nbgrader for my class mid-semester and it pretty much worked without a hitch (except for students who renamed their notebooks for no reason), so thank you for developing this wonderful tool!

[jhamrick]

Also, I deployed nbgrader for my class mid-semester and it pretty much worked without a hitch (except for students who renamed their notebooks for no reason), so thank you for developing this wonderful tool!

Thanks for the kind words 😄 I'm really glad it's been working out well for you!

[cjnitta]

I am looking for something similar to this, and didn't want to generate a whole new issue. Like @rgerkin situation we are looking at using nbgrader for a class where we assume the students have no programming experience. We would like to provide them with code and have them fix it. Right now we are thinking of doing something like:

broken_code = here
### BEGIN SOLUTION
fixed_code = here
### END SOLUTION

But this will generate:

broken_code = here
# YOUR CODE HERE
raise NotImplementedError()

Ideally we would have as output something like:

# FIX CODE BELOW
broken_code = here
# FIX CODE ABOVE
raise NotImplementedError()

Maybe the autograded answer would be:

### BEGIN GIVEN
broken_code = here
### END GIVEN
### BEGIN SOLUTION
fixed_code = here
### END SOLUTION

Or even something like:

### BEGIN SOLUTION
### BEGIN GIVEN
broken_code = here
### END GIVEN
fixed_code = here
### END SOLUTION

If there is a way to produce the output that I just totally missed, please point me to the documentation. Thanks for all the hard work you have put in, I think nbgrader will save us a lot of development time.

[Sefriol]

Usually in the simplest case we just comment out and example line which students could use as a reference.

#broken_code = 1 + ...
### BEGIN SOLUTION
broken_code = 1 + 2
### END SOLUTION

Which would turn into:

#broken_code = 1 + ...
# YOUR CODE HERE
raise NotImplementedError()

Or something really similar. Some very early coders in our courses did not even get raise NotImplementedError() i.e. they thought that there is an error in their code and we had to add a guidance to that as well.

I think idea proposed cjnitta, might make it more complicated to run instructor notebooks. If there we a good way to show this in UI (like a split div left side showing the instructor version and right showing the student version. Or like git diff view.), it might be a good idea. Otherwise we need even more time to put into teaching teachers how to make these notebooks. If it's clearly shown in UI, this teaching might not be required.

I would put this on hold until we can have an elegant way to deal with this.

[hoffm386]

I'm not sure if there are plans to implement this suggested improvement, but this is the workaround that has been working for my team. The workaround is # Replace None with appropriate code.

Examples

Basic Example

When creating a solution cell, we do something like this:

# Replace None with appropriate code
answer = None
### BEGIN SOLUTION
answer = 7
### END SOLUTION

Then the students see:

# Replace None with appropriate code
answer = None
# your code here
raise NotImplementedError

x_plus_one Example

It's also possible to assign part of an expression to None, rather than the whole thing. So the x_plus_one example would look like this in the assignment creation view:

# Replace None with appropriate code
x_plus_one = x + None
### BEGIN SOLUTION
x_plus_one = x + 1
### END SOLUTION

And like this in the student view:

# Replace None with appropriate code
x_plus_one = x + None
# your code here
raise NotImplementedError

Notes

Benefits

None is nice compared to using a keyword like pass that you might conventionally use as a placeholder, since it doesn't interrupt the execution order. Setting a value to None rarely has unintended side effects.

We think it's important to give a variable amount of "starter code" so that students aren't getting questions wrong based on silly things like spelling a variable name incorrectly. This approach lets us make fairly flexible starter code, sometimes with an entire line being just None, but still having an inline code comment helping them along, e.g.:

# Close the file
None

Sometimes a given code cell has just one None that they need to replace, sometimes it has multiple. You can even get clever by putting code that relies on the solution after the SOLUTION tags, e.g.:

# Replace None with appropriate code
X = None
y = None
### BEGIN SOLUTION
X = df.drop("target", axis=1)
y = df["target"]
### END SOLUTION
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

Then the students see this:

# Replace None with appropriate code
X = None
y = None
# your code here
raise NotImplementedError
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

Limitations

Learning to follow this pattern can tricky for students. They have to identify all instances of None, erase the None text, and write in the correct answer. (Unlike something like a textbox where you just click + type.) We do this frequently throughout the curriculum so students get used to it, but this might be too confusing of an interface for a one-off assessment.

Also of course, if the code actually needs to include None for some reason, you would need to change the instructions (and students might not notice).

Future Improvement Plan

It's still somewhat confusing to see # your code here when they may or may not actually need to write any code on that particular line. So we are going to try to modify the configuration so that instead of replacing the solution with this:

# your code here
raise NotImplementedError

Instead we replace the solution with this:

# Erase the following line when you have completed writing your solution in this cell
raise NotImplementedError

Then the student-facing version of the above example would be:

# Replace None with appropriate code
X = None
y = None
# Erase the following line when you have completed writing your solution in this cell
raise NotImplementedError
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

We haven't quite figured out how to implement this on my team yet (there are some inconsistencies right now in the documentation of how this attribute is named, and we are running the IllumiDesk fork of this project so there may be additional differences) but I'll report back here if we do get something like this working.

[perllaghu]

@hoffm386 - I like this advice..... alot

As for changing the # your code here bit, that's actually easy: add/create a local nbgrader_config.py file, and add the following:

c.ClearSolutions.code_stub = dict(
    python = '# Erase the following line when you have completed writing your solution in this cell\nraise NotImplementedError()',
    )

[ or, if you're being fancy - you can use a raw string:

c.ClearSolutions.code_stub = dict(
    python = r'# Erase the following line when you have completed writing your solution in this cell
raise NotImplementedError()',
    )

]

NOTE: If you're working with languages that don't use # for comments, then things are a bit more complicated:

c.ClearSolutions.code_stub = dict(
    stata = '* YOUR CODE HERE\nerror 192'
    )
c.ClearSolutions.begin_solution_delimeter = "* BEGIN SOLUTION"
c.ClearSolutions.end_solution_delimeter = "* END SOLUTION"

or

c.ClearSolutions.code_stub = dict(
    haskell = '-- YOUR CODE HERE\nerror "Not Implmented"'
    )
c.ClearSolutions.begin_solution_delimeter = "-- BEGIN SOLUTION"
c.ClearSolutions.end_solution_delimeter = "-- END SOLUTION"

(Sadly, you can't mix solution_delimiters)