olive-testing (TC_AUTHOR_45): Advanced problem types

[ARMBouhali]

Passing Tests

• [x] Blank advanced Problem
• [ ] Circuit Schematic Builder
• [ ] Custom JavaScript Display and Grading
• [ ] Custom Python-Evaluated Input
• [x] Drag and Drop
• [ ] Drag and Drop (Deprecated Version)
• [x] Image Mapped Input
• [x] Math Expression Input
• [ ] Problem with Adaptive Hint
• [x] Staff Graded Points

[ARMBouhali]

• [x] Studio problem display issue (affects LMS too)

Error formatting HTML for problem:
cannot create LoncapaProblem block- v1:FennX+Mehdi+Test+type@problem+block@13813106ad2f4d8da86d85936e03a371: 
Error while executing script code: [Errno 2] No such file or directory: 'TMPDIR=tmp'

encountered with:

• Custom JavaScript Display and Grading
• Custom Python-Evaluated Input
• Problem with Adaptive Hint

[ARMBouhali]

• [ ] Studio dispay issue for Drag and Drop (deprecated): Images are not showing.

[ARMBouhali]

• [x] LMS submit error:

Staff debug info: Traceback (most recent call last): File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2298, in execute_check_function safe_exec.safe_exec( File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner return func(*args, **kwds) File "/openedx/edx-platform/xmodule/capa/safe_exec/safe_exec.py", line 170, in safe_exec exec_fn( File "/openedx/venv/src/codejail/codejail/safe_exec.py", line 154, in safe_exec res = jail_code.jail_code( File "/openedx/venv/src/codejail/codejail/jail_code.py", line 318, in jail_code status, stdout, stderr = run_subprocess_fn( File "/openedx/venv/src/codejail/codejail/subproc.py", line 39, in run_subprocess subproc = subprocess.Popen( # pylint: disable=subprocess-popen-preexec-fn File "/opt/pyenv/versions/3.8.12/lib/python3.8/subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/opt/pyenv/versions/3.8.12/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'TMPDIR=tmp' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/openedx/edx-platform/xmodule/capa_module.py", line 1777, in submit_problem correct_map = self.lcp.grade_answers(answers) File "/openedx/edx-platform/xmodule/capa/capa_problem.py", line 436, in grade_answers new_cmap = self.get_grade_from_current_answers(answers) File "/openedx/edx-platform/xmodule/capa/capa_problem.py", line 494, in get_grade_from_current_answers results = responder.evaluate_answers(self.student_answers, oldcmap) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 314, in evaluate_answers new_cmap = self.get_score(student_answers) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2268, in get_score self.execute_check_function(idset, submission) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2309, in execute_check_function self._handle_exec_exception(err) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2524, in _handle_exec_exception raise ResponseError(text_type(err), traceback_obj) xmodule.capa.responsetypes.ResponseError: ("[Errno 2] No such file or directory: 'TMPDIR=tmp'", ) encountered with:

• Circuit Schematic Builder
• Drag and Drop (Deprecated Version)

[mariajgrimaldi]

Some of these issues could be solved by adding codejail in the sandbox, since it's not installed by default. I'll include it after I finish with olive support. I'll let you know :)

[ARMBouhali]

@mariajgrimaldi I've reproduced the same error in a tutor-nightly installation.

[Errno 2] No such file or directory: 'TMPDIR=tmp'

I could make the XBlocks work using the unofficial codejail service tutor plugin by eduNEXT . But it won't count as a success since codejail is already installed along with edx-platform and should work with no worries.

Same issue reported in the forum in the past https://discuss.openedx.org/t/errno-2-no-such-file-or-directory-tmpdir-tmp/5568

This probably would need intervention at Tutor's openedx Dockerfile template. I hope someone provides a fix without having to use a dedicated container as I did above.

[mariajgrimaldi]

While secure code execution is enabled, you'll need a sandbox with special permissions & setup to execute problems with custom code, this is what the codejail plugin offers. That's why is failing. In case you want to execute unsafe code, you'll need to add this to your settings: COURSES_WITH_UNSAFE_CODE = ["regex for courses with unsafe code"] Which needs to be added manually. Now, the demo is supposed to use the codejail plugin. I added it to the installation a couple of days ago, can you try again?

[mariajgrimaldi]

label: olive testing

[DeanJayMathew]

@ARMBouhali let me know if I can help test these now that codejail is added to the installation by @mariajgrimaldi.

[mariajgrimaldi]

As part of my codejail testing efforts, I tested:

• [x] Custom JavaScript Display and Grading
• [x] Custom Python-Evaluated Input
• [x] Problem with Adaptive Hint

https://user-images.githubusercontent.com/64440265/203297299-c382ac9f-88dc-4b90-8292-d78c905b01d0.mp4

[mariajgrimaldi]

There's an issue with the Circuit Schematic Builder, so I opened a PR that solves it 😅: https://github.com/openedx/edx-platform/pull/31365

[DeanJayMathew]

label: release blocker

[mariajgrimaldi]

Here's the PR to tutor olive branch fixing the template error.

Also, we got this ticket a few days ago in the codejail plugin, and I thought it was related to our latest upgrades. But after researching a bit more with my team, we realized we'd had that issue since the Lilac release, i.e, it has never worked 😅. We're working on it, but I wanted to let you guys know since it's related to the Advanced Blank Problem capabilities. Also, if you have any suggestions on how to solve it, we're all ears.

[ARMBouhali]

@mariajgrimaldi with this we have only the drag and drop (deprecated) image display issue in studio to clear the list.

I re-tested it now and I'm getting this screen:

And this console error:

drag_and_drop.a31124208b9b.js:8 Uncaught TypeError: requirejs.config is not a function
    at drag_and_drop.a31124208b9b.js:8:19
    at drag_and_drop.a31124208b9b.js:25:2

[mariajgrimaldi]

I wonder why are we testing a deprecated problem? 🤔

[mariajgrimaldi]

@arbrandes @DeanJayMathew: should we remove the release blocker label? IMO the issues left are not blockers

[arbrandes]

Let's ditch the deprecated (and clearly unsupported) drag-and-drop test, and call this done.