activitysim icon indicating copy to clipboard operation
activitysim copied to clipboard

Reproducible random does not work in trip scheduling choice

Open jpn-- opened this issue 1 year ago • 1 comments
trafficstars

Describe the bug The reproducible random code is supposed to ensure that, when run with a fixed seed, household-relevant results (persons, trips, tours) for any particular household remain stable no matter how many other households are in the sample. This is not the case for the trip scheduling choice model (currently used only by ARC).

Note this bug does not impact the statistical distribution or validity of results, it merely makes model testing problematic.

To Reproduce Steps to reproduce the behavior:

  1. Run the ARC model single process for some moderate sized household sample.
  2. Run the ARC model multi-process, with num_processes > 1 for the same household sample. Some households will be processes in different groups, which should not change the results.
  3. Observe that a small percentage of household trip schedules differ, typically by a small amount.
- Column 'main_leg_duration' failure
AssertionError: Series are different

Series values are different (22.66227 %)
[index]: [18413, 18448, 18473, 18489, 30371, 66957, 66992, 66998, 67033, 67066, 67104, 67148, 88747, 88763, 127869, 127871, 271675, 271676, 271683, 271746, 271779, 271820, 391140, 391142, 391179, 391214, 391226, 391261, 410255, 410256, 410359, 410378, 410443, 533935, 533941, 541799, 541846, 543947, 543986, 544019, 578877, 633385, 633407, 633437, 633442, 633489, 633522, 633530, 656794, 656812, 656853, 656854, 656867, 656892, 721604, 721609, 721639, 721669, 721713, 721754, 721795, 779015, 779028, 779066, 788961, 811612, 811620, 811631, 811664, 977162, 977230, 977233, 977260, 1031632, 1031673, 1178586, 1178601, 1178625, 1178636, 1178637, 1178660, 1178677, 1178740, 1224715, 1224750, 1224791, 1224792, 1224824, 1234344, 1234377, 1234418, 1254855, 1254856, 1254857, 1254912, 1254926, 1254956, 1254959, 1255000, 1274401, ...]
[left]:  [0, 20, 4, 21, 19, 3, 16, 1, 21, 23, 6, 11, 1, 24, 13, 2, 0, 1, 0, 19, 17, 23, 2, 2, 15, 2, 2, 18, 2, 2, 17, 31, 4, 1, 3, 4, 14, 5, 16, 26, 21, 0, 21, 1, 3, 17, 5, 22, 6, 3, 3, 2, 4, 21, 1, 0, 9, 2, 17, 24, 16, 8, 4, 4, 22, 4, 1, 7, 0, 0, 6, 18, 5, 15, 8, 9, 5, 10, 2, 4, 2, 0, 19, 3, 21, 20, 14, 16, 16, 16, 16, 4, 1, 1, 4, 24, 2, 15, 23, 15, ...]
[right]: [0, 20, 4, 19, 19, 3, 16, 1, 21, 23, 6, 11, 1, 24, 15, 2, 0, 1, 0, 20, 17, 23, 2, 2, 15, 2, 2, 18, 2, 3, 17, 31, 4, 1, 3, 4, 14, 5, 11, 26, 21, 0, 21, 1, 5, 14, 5, 22, 6, 3, 3, 1, 4, 18, 1, 0, 8, 2, 17, 24, 16, 8, 4, 4, 21, 4, 2, 7, 1, 0, 6, 20, 5, 15, 8, 9, 5, 13, 2, 4, 2, 0, 19, 3, 21, 20, 13, 16, 16, 16, 16, 4, 1, 1, 4, 24, 2, 15, 23, 15, ...]

Expected behavior Multiprocessing should not alter the results.

jpn-- avatar Sep 25 '24 21:09 jpn--