Messed up values in params sweep with BatchRunner

[manentai]

I have no previous experience with BatchRunner, so maybe I am missing something, but I have this params:

variable_params = {
    "population": [1000],
    "network" : ["random"],
    "average_degree": [4, 8, 16],
    "audit_perc": [0.01, 0.03, 0.1],
    "tax_perc": [0.1, 0.3, 0.5],
    "eta00": [0],
    "eta1": [1.2, 2, 3],
    "memory": [1, 5, 10]
}

and I am calling the BR like:

batch_run = BatchRunner(TEModel,
                    variable_params,
                    iterations=2,
                    max_steps=10,
                    model_reporters={"datacollector": lambda m: m.datacollector})
batch_run.run_all()
run_data = batch_run.get_model_vars_dataframe()
run_data.head()

And I get this df, where parameters sweep is not what it was expected:

It looks like column labels don't match with the column values... This is the params required in the init of my Model class:

def __init__(
    self, 
    population=1000, 
    network="random", 
    average_degree=4, 
    audit_perc=0.01, 
    tax_perc=0.1, 
    eta00=0, 
    eta1=1.2, 
    memory=1
):

Where am I missing something?

[tpike3]

That is a bug. Should be fixed in the next push, which should be relatively soon unless there is an issue in QA/QC.

[manentai]

@tpike3 thanks for the info, just as a confirmation: this a just a column label issue for the df, right?

[tpike3]

Correct, that is how I understand it

[tropappar]

I still experience this problem, even though I installed the GitHub version of Mesa using$ pip install -e git+https://github.com/projectmesa/mesa#egg=mesa

[tpike3]

@tropappar Hmmmm...my understanding is you want the datacollector to be all the items you are collecting during your model runs. What are you collecting with datacollector, I am not seeing that code?

As an example form the tutorial

self.datacollector = DataCollector(
            model_reporters={"Gini": compute_gini},
            agent_reporters={"Wealth": "wealth"})

Right now your model reporter is just returning the datacollector object.

[tropappar]

The problem is not about the data collector. Like @manentai explained, the problem is that the dataframe retrieved with run_data = batch_run.get_model_vars_dataframe() is messed up: The column headers corresponding to the variable parameters are sometimes not in the correct order.

[tpike3]

@tropappar Aww geez, I am sorry I totally missed that, I was rushing. Could you try calling batch_run.get_collector_model() instead this will return a dictionary with key = (Param1, Param,2...iteration), value=Pandas dataframe of collection.

Please let me know if that works for you.

[tropappar]

Yes, it works. But it does not solve the original problem. In fact, it gives a dictionary, so there are no headings for the prameters/keys. Thereby this solution simply avoids matching the parameter names (i.e. headings) to the parameter values (i.e. dict keys).

[tpike3]

@tropappar there must be a bug in the batchrunner function prepare_report_table(self, vars_dict, extra_cols=None). It will take me a week or 2 to untangle it... I imagine.

However as a stop gap you can take the dictionary output and turn that into a pandas dataframe so quick draft psuedoish code....

dict = {param1:[], param2:[]...}

for params, dfs in collector_model.items :
    param1.append(params[0])
   param2.append(params[1])
  .....

df = pd.DataFrame(dict)

I appreciate this isn't ideal but hopefully it gets you through the issue. If you happen to figure out the issue in prepare_report_table, I would be happy to review a pull request.

[jackiekazil]

You can track the next push here: https://github.com/projectmesa/mesa/pull/957

[tropappar]

Thanks for your feedback. Currently, I can live with this. I would assume the problem comes from the fact that the variable_params is a dictionary which is by definition not guaranteed to be ordered. Hence, sometimes it works and sometimes the order of the columns are wrong.