pymc-examples icon indicating copy to clipboard operation
pymc-examples copied to clipboard

Bayesian Hierarchical Stacking - well switching case study

Open MarcoGorelli opened this issue 4 years ago • 10 comments

Description

A Python implementation of the first case study from https://arxiv.org/pdf/2101.08954.pdf

MarcoGorelli avatar Sep 13 '21 16:09 MarcoGorelli

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

haven't really read the notebook yet, only 1 minute skim to check formatting mostly, I'll read the notebook whenever I have time, I love that we have a notebook for this paper :heart_eyes: . Some quick comments

The initial image doesn't render on the website, I suspect the reason for that is that raw html is not modifed by sphinx and the naive markdown/jupyter preview uses a different home folder than the sphinx/website renders do. It might be possible to get myst to modify raw html to fix this issue, I'd guess it will but I don't know. Whatever the case, https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#images and references therein will either explain how to get myst to handle this html image case or explain the recommended way of handling that. I think the ideal case would be to use the image directive and add a caption to the image.

The theme already generates a table of content automatically at the right sidebar, there is no need for a manual one too. You should also double check the title levels. While we are using markdown, sphinx only supports consecutive title levels which means that the main notebook title should be level 1, intro/eda/preparation... level 2 all of them, and the subsections should be level 3. Somehow this message Non-consecutive header level increase; 1 to 3 [myst.header] got to the notebook exerpt at https://pymc-examples--226.org.readthedocs.build/en/226/blog/tag/pymc3model.html.

There is one somewhat working citation (bibtex missing journal error only) and one non working myst citation. It looks like you the wrong one is using rST instead of myst syntax. Also, to ease thing going forward, it would be great to start the reference ids with the first author's last name and sorting them alphabetically so when it's a giant file we can still find which references are there and avoid duplication. And lastly, even though the reference is not working inline, the reference still appears in the bibliography, so it looks like the explanation at https://github.com/pymc-devs/pymc/wiki/PyMC3-Jupyter-Notebook-Style-Guide#references is not clear enough. I'll try to explain better here, if you can then fix the wiki that would be great. All references cited within the text with {cite:t/p/...} will automatically appear in the bibliography, this is what the filter takes care of. References only need to be listed manually within the bibliography directive if they are not cited inline but we still want them to appear in the bibliography.

OriolAbril avatar Oct 02 '21 13:10 OriolAbril

Thanks @OriolAbril for your feedback - I'm gonna be on holiday for the next two weeks, so marking as draft for now, will pick it up again when I'm back

MarcoGorelli avatar Oct 02 '21 13:10 MarcoGorelli

Nit: put variable names in the list before cell 4 in backticks to be consistent with what you have below.

fonnesbeck avatar Apr 02 '22 20:04 fonnesbeck

Is there any context as to why those 6 models were selected for comparison? If so, it would be helpful to add a line or two addressing this.

fonnesbeck avatar Apr 02 '22 20:04 fonnesbeck

There's no context unfortunately, and to be honest I think a more diverse set of models might have been more illustrative (e.g. on Kaggle the boost that people get from combining models is when the models are different, e.g. a GBDT and a neural network)

MarcoGorelli avatar Apr 02 '22 21:04 MarcoGorelli

Remove %matplotlib inline statement. This is no longer needed in Jupyter for inline plotting.

fonnesbeck avatar Apr 02 '22 23:04 fonnesbeck

Just noticed that all of this is in PyMC3/Theano. Since v4's release is imminent, it should be ported over.

fonnesbeck avatar Apr 02 '22 23:04 fonnesbeck

@MarcoGorelli will you be able to port this to v4, or do you want to hand it off?

fonnesbeck avatar May 11 '22 14:05 fonnesbeck

Hey - apologies, haven't had a chance to get back to this

If someone wants to take over, they're more than welcome to, that'd be great, thanks

Else I'll get back to it when I have a chance

MarcoGorelli avatar May 11 '22 14:05 MarcoGorelli

Else I'll get back to it when I have a chance

probably not gonna have a chance anytime soon - closing then, and I'm sorry about this

MarcoGorelli avatar Oct 09 '22 11:10 MarcoGorelli