PrairieLearn
Add check for duplicate answers to MC questions
One of the things I realized from reviewing a PHYS 10x question last week is that sometimes auto-generated distractors can repeat. I wonder if there's something we could do at the question generation level to inspect for this and take action (invalidate and make a new variant?)
That's not the same as duplicated manual answers, but shows up the same to the student.
Agree that would be nice, but it would be difficult to differentiate between "the question is fundamentally broken" and "just this variant is bad, we can regenerate". I'd prefer to just encourage people to a) be aware of that when and b) use "test 100 times" a lot to check for edge cases.
I actually think the right place to do this is in prepare(), so it will be caught very early.
I was thinking of taking a stab at this issue since it hits my students often in some ultra-randomized "sig fig" type questions where they have to choose the right answer with different errors.
IMHO, I think an error should be thrown if there are multiple choice options whose string representations are exactly identical. That way, students will see a regenerate variant button right away.
Does that make sense as the expected outcome?
IMHO, I think an error should be thrown if there are multiple choice options whose string representations are exactly identical. That way, students will see a regenerate variant button right away.
Yes, this sounds correct to me. I think my original comment was wrong and it's best to catch this in generate() and throw an error if any duplicates are found in the list of answers. Allowing students to regenerate a broken variant will often fix the problem.
it's best to catch this in
generate()and throw an error if any duplicates are found in the list of answers.
As commented on the PR for this, elements don't call the generate() function, so I think catching duplicates in prepare() (as is done currently in the PR) is the way to go.
