edsl

edsl copied to clipboard

Job running / progress bar enhancements

2

- [x] Batching of interview tasks to create smoother progress bar - [ ] Refactor report generation calling / remove failing question stuff - [x] Run jobs and progress bar...

johnjosephhorton

QuestionCheckBox and QuestionList validation

3

I can initialize QuestionCheckBox with negative values for `min_selections` and `max_selections` - I don't think this should be the case? ```python >>> from edsl import QuestionCheckBox >>> q = QuestionCheckBox(question_name="test",...

onmyraedar

I think they should all show "None" for consistency. _Originally posted by @rbyh in https://github.com/expectedparrot/edsl/issues/1063#issuecomment-2362072636_

johnjosephhorton

Need to handle `model` as a `Scenario` key

2

https://www.expectedparrot.com/content/21fe9f7d-16fd-422c-8784-5c72bea44dbb I want to be able to do this in a notebook comparing/scoring model responses, where `model` is a metadata scenario key.

rbyh

Question parameter `include_comment=False` not working for all types

7

Error thrown for linear scale, yes/no, budget, topk https://www.expectedparrot.com/content/4cc9e610-899c-410c-bb88-fd58d6cb9f64

rbyh

Accommodate models without system_prompts (like o1 class models) by not passing a system prompt if no Agent defined

1

johnjosephhorton

rbyh

Model report - used to find issues in models or questions

2

I created a report that is generated by iterating through all models and administers a survey consisting of all of our example questions. [model_report.txt](https://github.com/user-attachments/files/16922607/model_report.txt) This can be created by running:...

johnjosephhorton

Add skip rule capability for skipping a question in all cases

4

E.g., if I don't want to administer a question it would be nice for this to work (it does not -- there is no error, but the question is administered):...

rbyh

When the study json gets too big, can't read it in - often times there's lots of duplicate data in a study. @johnjosephhorton

benjamin-manning