Need for resetting in run som scripts

[sanjari-orb]

Hi, I am looking at the scripts run_reddit_som.sh, run_shopping_som.sh, run_classifieds_som.sh. IIUC, they all involve creating batches of indices and the docker gets reset between each of these batches.

https://github.com/web-arena-x/visualwebarena/blob/b56b6d821e0b0f926fb940a7efe7d3f1246eab36/scripts/run_reddit_som.sh#L21

However, I think more than one example with require_reset: True can occur in every batch based on the raw config JSON files (eg: https://github.com/web-arena-x/visualwebarena/blob/main/config_files/vwa/test_classifieds.raw.json).

If that is the case, what is the point of resetting and how are we ensuring correctness of the run scripts?