Problem with array indexing

[trondkr]

I am running the latest (Github version) of OpenDrift to simulate particles (these are simulations from Glomma using the MARTINI model) when I encounter a problem with indexing. The problem has occurred several times in the last few days of doing various simulations and at varying timesteps making it hard to debug. The problem seems to vary with how many particles and timesteps I use, but the error message is always something like the error message pasted below. I wonder if you have any suggestions as to what may go wrong ?

....

20:23:08 DEBUG   opendrift.models.basemodel: Adding uncertainty for wind: 2.0 m/s
20:23:08 DEBUG   opendrift.models.basemodel: ------------ SUMMARY -------------
20:23:08 DEBUG   opendrift.models.basemodel:     x_sea_water_velocity: 0.0727367 (min) 0.0727367 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     y_sea_water_velocity: -0.184277 (min) -0.184277 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_surface_wave_significant_height: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_ice_area_fraction: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     x_wind: 1.85206 (min) 1.85206 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     y_wind: 10.69 (min) 10.69 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     land_binary_mask: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_floor_depth_below_sea_level: 12 (min) 12 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     ocean_vertical_diffusivity: 3.6278e-06 (min) 3.6278e-06 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_water_temperature: 2.77071 (min) 2.77071 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_water_salinity: 16.1965 (min) 16.1965 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     surface_downward_x_stress: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     surface_downward_y_stress: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     turbulent_kinetic_energy: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     turbulent_generic_length_scale: 0 (min) 0 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     upward_sea_water_velocity: 0.00010178 (min) 0.00010178 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_surface_wave_stokes_drift_x_velocity: 0.01 (min) 0.01 (max)
20:23:08 DEBUG   opendrift.models.basemodel:     sea_surface_wave_stokes_drift_y_velocity: 0.01 (min) 0.01 (max)
20:23:08 DEBUG   opendrift.models.basemodel: ---------------------------------
20:23:08 DEBUG   opendrift.models.basemodel:            1 active elements
20:23:08 DEBUG   opendrift.models.basemodel:            latitude =  59.171531973212524
20:23:08 DEBUG   opendrift.models.basemodel:            longitude = 10.960446072517911
20:23:08 DEBUG   opendrift.models.basemodel:            z = -8.838237197396964
20:23:08 DEBUG   opendrift.models.basemodel: ---------------------------------
20:23:08 DEBUG   opendrift.models.physics_methods: Calculating Hs from wind, min: 2.895560, mean: 2.895560, max: 2.895560
20:23:08 DEBUG   opendrift.models.basemodel: No elements hit coastline.
20:23:08 DEBUG   opendrift.models.basemodel: No elements hit seafloor.
20:23:08 DEBUG   opendrift.models.basemodel: Writing history for 1 deactivated elements
20:23:08 DEBUG   opendrift.models.basemodel: Removed 1 elements.
20:23:08 DEBUG   opendrift.models.basemodel: Removed 1 values from environment.
20:23:08 DEBUG   opendrift.models.basemodel: remove items from profile for z
20:23:08 DEBUG   opendrift.models.basemodel: remove items from profile for sea_water_temperature
20:23:08 DEBUG   opendrift.models.basemodel: remove items from profile for sea_water_salinity
20:23:08 DEBUG   opendrift.models.basemodel: remove items from profile for ocean_vertical_diffusivity
20:23:08 DEBUG   opendrift.models.basemodel: Removed 1 values from environment_profiles.
20:23:08 INFO    opendrift.models.basemodel: No active elements, skipping update() method
20:23:08 DEBUG   opendrift.models.basemodel: Horizontal diffusivity is 0, no random walk.
20:23:08 DEBUG   opendrift.models.basemodel: 0 active elements (555 deactivated)
20:23:08 DEBUG   opendrift.models.basemodel: to be seeded: 445, already seeded 555
20:23:08 DEBUG   opendrift.models.basemodel: Released 0 new elements.
20:23:08 INFO    opendrift.models.basemodel: No active but 445 scheduled elements, skipping timestep 2398 (2019-03-17 15:30:00)
20:23:08 DEBUG   opendrift.models.basemodel: to be seeded: 445, already seeded 555
20:23:08 DEBUG   opendrift.models.basemodel: Released 0 new elements.
20:23:08 INFO    opendrift.models.basemodel: No active but 445 scheduled elements, skipping timestep 2399 (2019-03-17 15:40:00)
20:23:08 DEBUG   opendrift.models.basemodel: to be seeded: 445, already seeded 555
20:23:08 DEBUG   opendrift.models.basemodel: Released 0 new elements.
20:23:08 INFO    opendrift.models.basemodel: No active but 445 scheduled elements, skipping timestep 2400 (2019-03-17 15:50:00)
20:23:08 WARNING opendrift.models.basemodel: The simulation stopped before requested end time was reached.
20:23:08 INFO    opendrift.models.basemodel: ========================
20:23:08 INFO    opendrift.models.basemodel: End of simulation:
20:23:08 INFO    opendrift.models.basemodel: index 100 is out of bounds for axis 1 with size 100
20:23:08 INFO    opendrift.models.basemodel: Traceback (most recent call last):
  File "/cluster/projects/nn9297k/opendrift/opendrift/models/basemodel.py", line 2385, in run
    self.state_to_buffer()  # Append status to history array
  File "/cluster/projects/nn9297k/opendrift/opendrift/models/basemodel.py", line 2601, in state_to_buffer
    self.history[var][ID_ind, time_ind] = \
  File "/cluster/home/trondk/.conda/envs/opendrift/lib/python3.9/site-packages/numpy/ma/core.py", line 3381, in __setitem__
    _data[indx] = dval
IndexError: index 100 is out of bounds for axis 1 with size 100

20:23:08 INFO    opendrift.models.basemodel: 'The simulation stopped before requested end time was reached.'

20:23:08 INFO    opendrift.models.basemodel: ========================
20:23:08 DEBUG   opendrift.models.basemodel: Cleaning up
Traceback (most recent call last):
  File "/cluster/projects/nn9297k/Glomma_particles/run_sedimentdrift.py", line 112, in <module>
    main()
  File "/cluster/projects/nn9297k/Glomma_particles/run_sedimentdrift.py", line 108, in main
    run.start_simulations()
  File "/cluster/projects/nn9297k/Glomma_particles/run_sedimentdrift.py", line 97, in start_simulations
    self.create_and_run_simulation()
  File "/cluster/projects/nn9297k/Glomma_particles/run_sedimentdrift.py", line 77, in create_and_run_simulation
    o.run(end_time=self.confobj.end_date,
  File "/cluster/projects/nn9297k/opendrift/opendrift/models/basemodel.py", line 2492, in run
    self.state_to_buffer()  # Append final status to buffer
  File "/cluster/projects/nn9297k/opendrift/opendrift/models/basemodel.py", line 2601, in state_to_buffer
    self.history[var][ID_ind, time_ind] = \
  File "/cluster/home/trondk/.conda/envs/opendrift/lib/python3.9/site-packages/numpy/ma/core.py", line 3381, in __setitem__
    _data[indx] = dval
IndexError: index 100 is out of bounds for axis 1 with size 100

[knutfrode]

It is hard to see from this log what might be the problem. But since it is an indexing problem, it could be that the model-update() method is doing something wrong. Is this the "built in" sediment module, or some other module?

Would you be able to make a small script that reproduces this problem?

[trondkr]

This is a modified version of the "built-in" sediment module that accounts for calculating terminal velocity based on diameter and density. It could very well be something goes wrong there, but when I increased my time-step to 1 hour (I know it's very long...) the model runs correctly without the array indexing problem.

I can probably make a small example after Easter although the full module is available here: https://github.com/trondkr/Glomma_particles/blob/master/sedimentdrift.py Thanks!