randomize_order produces unexpected results

[matanmazor]

Dear Josh and team,

I'm using JsPsych for all my online experiments (🙏), and today for the first time I noticed an unexpected behaviour: when using 'randomize_order' to randomize trial order within a block, randomization seems to constrain the max number of consecutive trials in a row to 2. I always thought randomizing this way should result in an entirely pseudo-randomized trial order, without any constraints.

I was wondering if there's a way to relax this constraint, and whether this behaviour is expected/documented (I couldn't find it in the docs but maybe I'm searching in the wrong place).

Here's a code snipped that should reproduce this behaviour, adapted from the demo experiment in the tutorial:

var timeline = [];

  /* define fixation and test trials */
      var number = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: jsPsych.timelineVariable('stimulus'),
        trial_duration: 50
      };

      var test_stimuli = [
            { stimulus: "1"},
            { stimulus: "2"}
          ];

      /* define test procedure */
      var test_procedure = {
        timeline: [number],
        timeline_variables: test_stimuli,
        repetitions: 50,
        randomize_order: true
      };
      timeline.push(test_procedure);

(the number of switches between 1 and 2 is around 75% and not 50% as expected by chance alone).

Many thanks!! -Matan

[jodeleeuw]

Hi @matanmazor,

The repetitions parameter causes the entire timeline to repeat. On each repetition, a new random order will be generated when randomize_order is true. So with this configuration you will always get blocks of two trials where each block contains 1 and 2, though 1 will come before 2 ~50% of the time in each block.

The way to do what you are after is using the sample parameter, and specifically the fixed-repetitions option. The documentation for this option contains a helpful example.

Cheers!

[matanmazor]

Thanks @jodeleeuw, that's very helpful. I will make sure to change this for my future studies, and let my current and former labmates know (I think they also used the randomize_order approach). I wonder if it would be helpful to flag this in the "Demo Experiment: Simple Reaction Time Task", since I imagine many people would start there and adapt the experiment to their needs step by step. Maybe also worth a tweet? Again many many thanks!

[jodeleeuw]

@becky-gilbert if you are still working on docs updates what do you think about adding some clarification to the tutorial to cover this point?

[becky-gilbert]

@matanmazor! sorry about the confusion over how this works, and thanks for flagging it!

There's now a note about this in the documentation Simple RT Task tutorial. It's in the box called "Info: Randomizing timeline variables".

I didn't wait for feedback on this new info box because I was making some other docs changes alongside this one that I wanted to get out quickly. But if you have suggestions for improving the docs on this, please let us know!

[becky-gilbert]

Also @jodeleeuw @bjoluc you might've noticed this already, but in case not: I also made a few other changes to the RT Task docs alongside this one:

• added titles to the info boxes
• made the info boxes collapsible, and initially rendered them collapsed
• fixed a info box formatting problem
• fixed a few typos and links

See 540046b108e26e32a5500a1eb5d3fb1391341f17.

I went ahead with these changes because I figured they were pretty uncontroversial, but just let me know if you want me to change anything, or feel free to edit directly.