edsl

edsl copied to clipboard

Two integration tests failing for rank and yes/no questions

1

@rbyh could you investigate? ``` integration/test_questions.py::TestAllQuestions::test_question_rank FAILED [ 84%] integration/test_questions.py::TestAllQuestions::test_question_top_k PASSED [ 92%] integration/test_questions.py::TestAllQuestions::test_question_yes_no FAILED [100%] ```

johnjosephhorton

Certain Sphinx settings in conf.py not rendering properly: "sourcelink" should not appear: Relevant setting: "version" number should not appear: Relevant setting:

rbyh

Getting a validation error for `QuestionNumerical`, most likely because the model is ignoring the directions and trying to return a textual response. We should consider how to handle this case...

rbyh

For completeness, question memory should include question options that were presented

1

For completeness, question memory should include the question options that were presented, together with the question text and answer. Eg, here a multiple choice question was added as memory for...

rbyh

QuestionFunctional answer elements aren't getting correctly resolved into Result fields

1

johnjosephhorton

@apostolosfilippas It looks like this is farther along than I thought - I don't think we necessarily need changes on coop side. This is my current understanding: In https://github.com/expectedparrot/edsl/blob/main/edsl/jobs/runners/JobsRunnerAsyncio.py there...

johnjosephhorton

rbyh

rbyh

This is first step to making Jobs coopable.

johnjosephhorton

Add notion of "sections" to Survey object; allow section-specific skip-logic; Create Survey CSS

3

johnjosephhorton