mesa icon indicating copy to clipboard operation
mesa copied to clipboard

Update into_tutorial

Open tpike3 opened this issue 1 year ago • 7 comments

  • Update intro tutorial to be compatible with Mesa 3.0

tpike3 avatar Oct 16 '24 10:10 tpike3

A first quick comment. You currently have


class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N):
        super().__init__()

This should be changed to

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N, seed=None):
        super().__init__(seed=seed)

The same change should be made in all other places where you have the MoneyModel class. This ensures that seed is handled correctly and is controllable.

quaquel avatar Oct 16 '24 12:10 quaquel

A first quick comment. You currently have

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N):
        super().__init__()

This should be changed to

class MoneyModel(mesa.Model):
    """A model with some number of agents."""

    def __init__(self, N, seed=None):
        super().__init__(seed=seed)

The same change should be made in all other places where you have the MoneyModel class. This ensures that seed is handled correctly and is controllable.

I added seed with None and added a challenge to set a seed and see the impact when setting up the initial model and agent interaction. (In this case it demonstrates that the agent order stays in whatever random configuration the seed is set too)

As the seed kwarg is set to none, I am thinking we should not put it in each one. Thoughts?

tpike3 avatar Oct 17 '24 11:10 tpike3

Maybe we need to make seeds / random numbers a separate section - but not sure that fits in an introductory tutorial.

EwoutH avatar Oct 17 '24 11:10 EwoutH

As the seed kwarg is set to none, I am thinking we should not put it in each one. Thoughts?

Maybe we need to make seeds / random numbers a separate section - but not sure that fits in an introductory tutorial.

  1. Seed analysis is critical in developing and using ABMs
  2. So, we need to ensure that all examples ensure that the seed is user-controllable (hence my original comment) even if it's not used in each example.
  3. Boltzman is not an ideal model to showcase the impact of changing the seed. Something like wolf-sheep might be better for that because you can get 3 distinct dynamics: wolves eating all sheep and then also dying out, wolves dying out and sheep taking over (limited by grass), and meta-stable dynamic equilibrium between wolf and sheep over many steps. There might thus be room for a further tutorial, probably centered on experimentation (so also covering batch run and something very basic on the design of experiments). Here we might explicitly cover seed analysis.

quaquel avatar Oct 18 '24 06:10 quaquel

I think it is critical to have the seed in there and upfront, it is one of those oft overlooked and but critically important. Everyone wants to ignore the butterfly effect, because it is intuitively scary.

How about I bookend seed--- right up front and then as the last section is batchrun put it in there?

I got the writing part of the documentation done,

#TODO is add some of the new agentset functionalities.

tpike3 avatar Oct 18 '24 10:10 tpike3

How about I bookend seed--- right up front and then as the last section is batchrun put it in there?

Sounds good to me. We can keep an experimentation tutorial for later.

quaquel avatar Oct 18 '24 10:10 quaquel

Also please check for changes between the Model in the tutorial and the updated example in mesa/examples. The latter for example uses "n" instead of "N" for the number of agents

Corvince avatar Oct 18 '24 20:10 Corvince

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -2.7% [-4.9%, -0.4%] 🔵 -1.6% [-1.8%, -1.5%]
BoltzmannWealth large 🔵 +0.5% [+0.1%, +0.8%] 🔵 -2.7% [-4.1%, -1.2%]
Schelling small 🔵 -0.2% [-0.7%, +0.3%] 🔵 -1.3% [-1.6%, -1.0%]
Schelling large 🔵 +0.6% [-0.2%, +1.3%] 🔵 -2.0% [-3.1%, -1.1%]
WolfSheep small 🔵 -1.0% [-1.3%, -0.7%] 🔵 +0.2% [+0.0%, +0.4%]
WolfSheep large 🔵 -2.3% [-3.4%, -1.4%] 🔵 -3.7% [-6.8%, -1.2%]
BoidFlockers small 🔵 -1.7% [-2.5%, -0.9%] 🔵 +0.4% [-0.4%, +1.3%]
BoidFlockers large 🔵 -0.3% [-1.0%, +0.4%] 🔵 -0.6% [-1.0%, -0.1%]

github-actions[bot] avatar Oct 20 '24 11:10 github-actions[bot]

I am scanning the tutorial. A few quick points:

  1. There is a section titled "Making the Agents do", but we have a step function on the model and the agent, so this section probably needs a bit of a rewrite.
  2. In the "Making the Agents do" there is a link to the how to guide, but this is very outdated (see #2385).
  3. The section titled "Running the Model" says that running the model will result in printing the agent unique_id, but since there is no step method in MoneyAgent at that point, that's not correct.
  4. In the batchrun part there is still a mention of schedulers.

quaquel avatar Oct 20 '24 12:10 quaquel

Ok this is ready now that the build is working . As there is a lot of happies to glads, besides making compatible with 3.0 the three big parts to look at are:

1- I added an AgentSet functionality section. As I am always behind, I am a little worried I did code that worked vs good coding practices we want to instill in students 2 - I added a seed section at the beginning and then in the comparison portion of batch run with a quick overview of reproducibility and sensitivity analysis 3 - In the visual I made each part self contained. As the more dashboards being run the slower it gets. I also moved the stand alone histogram to after the last dashboard. This isn't elegant but I could not find an elegant way to end the process in one one cell with another cell.

This should however give us a very nice tutorial for CSSSA

tpike3 avatar Oct 20 '24 12:10 tpike3

I am scanning the tutorial. A few quick points:

  1. There is a section titled "Making the Agents do", but we have a step function on the model and the agent, so this section probably needs a bit of a rewrite.
  2. In the "Making the Agents do" there is a link to the how to guide, but this is very outdated (see The How-To guide is outdated #2385).
  3. The section titled "Running the Model" says that running the model will result in printing the agent unique_id, but since there is no step method in MoneyAgent at that point, that's not correct.
  4. In the batchrun part there is still a mention of schedulers.

Thanks--- on it!

tpike3 avatar Oct 20 '24 12:10 tpike3

This should however give us a very nice tutorial for CSSSA

I agree, this looks like a great tutorial. Once the few points I had are addressed this is good to go as far as I am concerned.

quaquel avatar Oct 20 '24 12:10 quaquel

I will review today!

EwoutH avatar Oct 20 '24 16:10 EwoutH

I am scanning the tutorial. A few quick points:

  1. There is a section titled "Making the Agents do", but we have a step function on the model and the agent, so this section probably needs a bit of a rewrite.
  2. In the "Making the Agents do" there is a link to the how to guide, but this is very outdated (see The How-To guide is outdated #2385).
  3. The section titled "Running the Model" says that running the model will result in printing the agent unique_id, but since there is no step method in MoneyAgent at that point, that's not correct.
  4. In the batchrun part there is still a mention of schedulers.

Thanks--- on it!

Updates made

  1. I updated the "Making Agents Do" section
  2. Part of the pull request is updating the "How To" to add agentset features (just taken form @EwoutH migration guide
  3. Fixed
  4. Removed

tpike3 avatar Oct 20 '24 19:10 tpike3

Really great job on the tutorial! I especially like all the challenges to get users to engage with the code!

Please check if the changes to the viz tutorial are intentional and as expected, but otherwise, good to go!

The repetition is intentional; although very brute force

The problem is when you run three dashboards at the same the last one barely moves. So each part is self contained with instructions to restart the kernel. So far I haven't found a good solution, with but this was better than users getting frustrated with the visuals. I imagine our tutorial is a bit of an edge case for jupyter and solara.

tpike3 avatar Oct 21 '24 09:10 tpike3

@Corvince or @maartenbreddels any suggestions? It's about this changeset in the visualisation tutorial (rendered: old | new).

EwoutH avatar Oct 21 '24 09:10 EwoutH

Really great job on the tutorial! I especially like all the challenges to get users to engage with the code!

Please check if the changes to the viz tutorial are intentional and as expected, but otherwise, good to go!

Awesome!! Thanks @EwoutH Very curious to see their thoughts!

tpike3 avatar Oct 21 '24 16:10 tpike3