PrairieLearn
element to hide in mode:exam
Questions sometimes include reference links that need the Internet that do not work in Internet-limited testing centers.
We should modify pl-hide-in-panel or make a similar element to hide content when "mode": "Exam" and encourage instructors to update their questions appropriately.
Here's another aspect to consider:
This would also be useful for questions where instructors want to include extra hints except at exam time. However, that benefit would be lost for students taking remote/async exams that aren't in Exam mode. (I know exam mode is different from exam type, but for reference, see @nwalters512's objection here: https://github.com/PrairieLearn/PrairieLearn/discussions/6108#discussioncomment-3321833 ) So if there's a risk that instructors might try to use this for anything other than marking up network-requiring items, it could be an antipattern.
Perhaps an alternative approach could be that individual questions that need external internet access could explicitly (but optionally?) declare this in their info.json, and then an Exam mode assessment would throw an error (or at sync time?) if any such questions are included. I'm thinking the declaration in the question's json wouldn't actually toggle network access, it would only assist in this kind of static checking at the assessment level.
To avoid confusion about what such a property in info.json actually does, perhaps it could be named something like examModeCompatible (default true) rather than mentioning "allow internet" or "require internet" in the name. However instructors would be need to be taught to set this to false if their question requires external net access, which may also be tough to communicate.
My main motivation is to stop unavailable links from showing up in questions in the CBTF. The students think they're getting short changed because they can't reference something in the question, they complain to the proctors, there's troubleshooting about what's wrong, etc. I just want to make that easier for all.
I can't see having faculty having the presence of mind to add more things into info.json, and then have to do things with it. I'm doubtful they would even use a <pl-hide-in-mode> tag unless we harass them about it.
It's probably easiest (?) to somehow parse the question at view time and put a warning/pop-up/something on the links that we know are blocked (i.e. not PrairieLearn) so students at least know the missing link is part of the question design not the CBTF misbehaving.
This is relevant again in some current discussions about what's visible in exams. Currently data used with the elements and external graded doesn't have any information about the environment in which the question is run. I think adding data["mode"] would be useful in this context and hopefully not too controversial (as adding other information into the question visibility, like who is taking it or which assessment it's included in, etc.)
Where in the code does the data dict that's shipped to the question server.py, element controller python, and external grader data.json live? (I'm hoping one place but I wouldn't be surprised if it's three....)
@trombonekenny see here for the data to be passed to render():
https://github.com/PrairieLearn/PrairieLearn/blob/7b955833c6802454e7f9121fd69efc31853bcb4f/apps/prairielearn/src/question-servers/freeform.js#L1139-L1154
TBD if we need this for prepare() and generate() too. I'm not convinced we need to provide this information to external graders; do you have a use case in mind?
Rather than data["mode"], consider data["assessment_mode"] or data["assessment_access_mode"] (bikeshed away). I'd rather namespace this.
Be sure to think through how to define the "mode" when the question is viewed outside the context of an assessment.
I'm still very skeptical about giving questions access to mode. The whole concept of mode is something that's already confusing, and having questions behave differently based on this mode scares me. I'm also not sure what the use case is here?
My main motivation is to stop unavailable links from showing up in questions in the CBTF. The students think they're getting short changed because they can't reference something in the question, they complain to the proctors, there's troubleshooting about what's wrong, etc. I just want to make that easier for all.
...
It's probably easiest (?) to somehow parse the question at view time and put a warning/pop-up/something on the links that we know are blocked (i.e. not PrairieLearn) so students at least know the missing link is part of the question design not the CBTF misbehaving.
I think this is legit. But if this is the only use case, I imagine we could do this by intercepting click events client-side and prompting students with a modal explaining access may be blocked if they're in exam mode. I haven't yet thought through exactly how good of an idea that'd be, but it's an option and would require zero intervention from instructors.
Isn't this effectively a more limited version of this existing PR?: https://github.com/PrairieLearn/PrairieLearn/pull/3848 This is effectively wanting questions to behave differently when given different injectable parameters (except here the parameter is just the assessment context and is not set manually). This also couples the mode to the question code itself, and I'm not sure what the behavior should be if the assessment types ever get changed (which is planned at some point in the future, if I remember right).
I'm also in agreement with @mwest1066 that having different static question behavior has the potential to cause issues, and it would be nice to have some feature to aid question development with respect to this.
@eliotwrobson this is access mode, not assessment type. That said, I think linking this to the assessment via params as Eliot mentioned is probably a better idea. I think it is a bad idea for students to see different question text depending on mode. If the difference is between different assessments it makes sense, but within the assessment the student should probably always see the same question text.
Let my XY this a bit. I think we might be able to do it with CSS and change the display based on context (without doing it as an element, which prompted this question).