Cannot save 3-dimensional outputs

[steipatr]

When using save_results to store the outputs of experiments on a ReplicatorModel, the following error appears:

ValueError: Must pass 2-d input. shape=(100, 3, 2)

The results are incompletely saved and cannot be loaded with load_results, although manual inspection shows that at least experiments.csv is tarballed correctly.

The shape of ReplicatorModel outputs is (n_experiments, n_replications, len_output). I imagine save_results is expecting only (n_experiments, len_output), so the added dimension of the replications throws it off.

I am using Workbench version 2.2 via Jupyter Lab.

I do not remember this being a problem previously, so it may be related to changes made in response to #94 ? We already had some discussions on making saving/loading results smoother back then.

I have attached a minimum working example: 3d save test.zip

[quaquel]

swap out the time series outcome for an ArrayOutcome and it should work just fine.

[steipatr]

That works, thanks! So this is intended?

[quaquel]

yes, but not well documented.

Basically, an ArrayOutcome is for any n-dimensional results. TimeSeriesOutcome subclasses ArrayOutcome and will receive at some point a meaningful index and most likely become a dataframe.

[steipatr]

Minor word of warning, if you try to apply the fix described above (ArrayOutcome instead of TimeSeriesOutcome for ReplicatorModels so the outcomes can be saved with save_results) to a NetLogoModel, the individual outcomes will have shape (n_experiments, n_replications) and contain only the last timestep. This might in some way be related to #89 , where we also discussed how the Workbench handles time series from NetLogo.

As I need the model behavior over time, I will stick with TimeSeriesOutcomes and see how convenient the pickle package is for saving and loading results.

[quaquel]

well that is clearly a bug then. The bug is in BaseNetLogoModel

    @property
    def ts_output_variables(self):
        if self._ts_output_variables is None:
            timeseries = [o for o in self.outcomes if isinstance(o, TimeSeriesOutcome)]

            self._ts_output_variables = [
                var for o in timeseries for var in o.variable_name
            ]

        return self._ts_output_variables

I guess that, instead of checking on TimeSeriesOutcome we should check on ArrayOutcome, and then all would be well.

[EwoutH]

@steipatr The issue you were experiencing should be resolved by #183! Could you verify that this solution works for you?

You can install the current master version of the EMAworkbench with Pip, using:

pip install -U -e git+https://github.com/quaquel/EMAworkbench #egg=ema-workbench