update Model Averaging example to run on pymc v5 #67

[alporter08]

update Model Averaging example to run on pymc v5

https://github.com/pymc-devs/pymc-examples/issues/67

• [x] Notebook follows style guide https://docs.pymc.io/en/latest/contributing/jupyter_style.html
• [x] PR description contains a link to the relevant issue:
  • a tracker one for existing notebooks (tracker issues have the "tracker id" label)
  • or a proposal one for new notebooks
• [x] Check the notebook is not excluded from any pre-commit check: https://github.com/pymc-devs/pymc-examples/blob/main/.pre-commit-config.yaml

Helpful links

• https://github.com/pymc-devs/pymc-examples/blob/main/CONTRIBUTING.md

---

:books: Documentation preview :books:: https://pymc-examples--600.org.readthedocs.build/en/600/

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[alporter08]

Hi @OriolAbril @cluhmann , I will look over the text to make sure it syncs with the code changes and look at the Jupyter Style guide. I also noticed there is an issue currently with the colors and axes in the final plot so will investigate that.

[cluhmann]

Great, thanks again for your the help!

[alporter08]

@OriolAbril @cluhmann I got the code to work with the new PyMC version, and the results match the old notebook. I also edited some of the text for clarity.

One point to mention however is that the notebook is supposed to demonstrate that the weighted model approach has more uncertainty than an individual model. But looking at the hdi, it is actually pretty much equal for the weighted approach and the compared individual model. I think this is because the model we are comparing the ensemble method to has a weight of one, and the other two models are effectively zero weights. So the ensemble method is really just one model, which we are comparing to itself. So maybe that should be specified in the commentary? Right now it says that the weighted model uncertainty is larger than the single model uncertainty, which isn't really the case.

[OriolAbril]

Given this is taken from an example in the book, we should reproduce it as it is. Then if we deem necessary, add extra comments. I have just checked, and the book uses the pseudo-BMA method to compute the weights. We should then use the method argument when calling az.compare, and use those weights to generate the weighted posterior predictive samples.

Afterwards, it is probably a good idea to use az.compare with the stacking method and use the results+reference to the section on stacking to explain why it is ArviZ's default now. What do you think? Tagging also @aloctavodia who authored the notebook

[aloctavodia]

I think is ok to merge, the mechanics is ok, even when the example itself is not that good. I can later work on a new version that better highlights the difference between doing model averaging and picking a single model.