gdsbook
CHAPTER: Spatial Inequality
While working on this chapter, I've noticed that the TheilD chapter works on the whole time series matrix (as does the Theil estimator) but the Gini estimator requires one-dimensional input.
This is a user experience/api inconsistency for giddy, I think?
I've gotten all the way through the Theil decomposition at this point. There's a lot of text that needed adding, and quite a bit that still needs to be added. It gets very sparse once the decomposition methods start.
I intend to remove: - the state-based inequality theil decomposition and subsequent exposition for state-within-region parts of the theil section - the spatial 20:20 section - change the rank path section to just illustrate the idea of rank paths.
Since there's just not enough space to explain them fully here. Alternatively, I could double down on the spatial 2020 + spatial 2020 rankpaths, but I think that's less informative?
Notes from DAB review:
- [ ] @ljwolf brings this up so will replicate here so we discuss:
we (1) don't discuss wide vs. long anywhere else in the book and (2) don't ever show a wide-to-long pivot using something like pandas.wide_to_long... We also don't need the LineCode filtering, since the final dataframe is already filtered by LineCode so that all LineCode==3.
- [ ] Related comments by DAB:
- On (1), I'd say that's OK, as we have it now still feels useful and not too out of context. I think (2) would be good but I don't see it as a crucial USP of the book.
- do we need the 28 extra columns if the analysis is univariate? If not, I'd vote to remote them to slim down the data footprint.
- [ ] My suggestion would be to reproject the data and leave it ready for analysis on the data cleaning process, not halfway the analysis
- [ ] We should agree on a global style for plots (this chapter uses seaborn's
whitegridbut I don't know if it's used anywhere else) - [x] ~~I'd consider whether we could shorten the development of the Gini index. It's a pretty common, non-spatial measure and currently takes quite a bit of space~~
- [ ] There are imports in halfway the chapter, when a new section requires them. Do we have a common policy for this across the book? Do we want one?
- [ ] If we need to trim a bit of space, the first comparison of Theil and Gini could be subsumed to that with Moran's I (we're repeating the same plots several times in expensive paper estate)
- [x] ~~If I get it right, the "Personal Vs Regional" section does not have any code. Is this on purpose? I'm not sure how it plugs into the broader outline of the chapter. To discuss.~~
- [ ] I'd propose to add the region names to the data prep section rather than intersected in the middle of the chapter, I think it distracts the reader.
- [ ] The function for applying the spatial Gini needs some clarification I have no idea about (potentially Ux expansion in the library itself?)
- [ ] I think the conclusion needs a bit more love
- [ ] I've not had a chance to add questions
OK my review is in at #134. I've done quite a bit of editting and I think we can merge and then move forward.
Before the chapter is considered ready for the publisher, here're the steps I'd suggest:
- @sjsrey, @ljwolf and @darribas to meet and discuss the actions above (maybe we don't want to do all of them, or not exactly as proposed).
- My suggestion is for @sjsrey to have a final go at it before we consider it done. Both Levi and I have changed it substantially and I think Serge's expertise is the ideal to make sure we're not missing anything and we're rounding it up as a chapter that noone else could do better.
To do, with/before final edits pass:
- [x] Remove incremental time series plots --> show individual ones and only one comparison at the end of the chapter
- [x] Adapt narrative text to reflect the above
- [x]
lorenz_curves.head()-->lorenz_curves.T.head()so it reduces the amount of printing - [ ] Broken math in Theil's index
- [x] Broken footnote on spatial Gini
@sjsrey is on the first final pass. To complete:
Dani final pass tracker, to be edited as we go along:
Things to ensure (from #152 ):
- [x] Library imports (first time inline, afterwards at the top of the chapter
- [x] No aliases on imports (
import pandas) - [x] Ensure no
XXXfor citations - [x] Ensure
matplotlibRC file is picked up and figures are standard - [x] Remove
%matplotlib inline - [x] Graphics
- No style
- No title (unless multi-axis titles, but not
suptitle) - No axes for maps (
ax.set_axis_off())
- [ ] Code comments are fine if needed
- [ ] Hide/remove code cells that we want to hide (see docs here)
- [x] Typesetting check
- [ ] General flow of the text
- [x] Ensure there is a "next step" section at the end of the chapter pointing to further references for the topic (re. #137)
Just going over #191 I'm not sure the bits highlighted in my previous comment have been incorporated?
https://github.com/gdsbook/book/issues/22#issuecomment-844221279
Just flagging to make sure we get them in before the three passes (e.g. removing figures with the same plots)
Bits from @darribas last pass:
- [x] Question for @sjsrey , @ljwolf and @darribas : do we want to mention dynamics in the title?
- Yes, to be included in the title by the next one
- [X] GENERAL: How are we dealing with figures inserted in the PDF detached from the code that generates them? At the very least, we need to make a clear statement at the beginning of the book. Ideally, each code cell generating a figure would print the Figure name that generates (or something like that).
- Yes, moved over to https://github.com/gdsbook/foundry/issues/44
- [x] Do we want refs for the Gini and Theil indices?
- Yes, @sjsrey to add them
- [X] GENERAL: we need a consistent way to name pysal/PySAL/Pysal across the book
- Decided over at https://github.com/gdsbook/book/issues/213
- [x] Do we have a citation for the decomposed Theil @sjsrey ?
- Yes, @sjsrey to add it
- [x] First equation in Section 10.5.3 (p.205) does not have a description of elements (what is $n$ and $\bar{x}$?)
- Yes, @sjsrey to add it
PR with the rest of edits over at https://github.com/gdsbook/book/pull/207