Package for analysing and visualising Australian election data

[benmarwick]

I just read about this new R package for NZ election data that looks pretty neat: http://ellisp.github.io/blog/2016/04/03/nzelect1/ & https://github.com/ellisp/nzelect/blob/master/README.md

It got me thinking that something similar for Australia could be a good open data project for this unconf. We could go further with some spatial viz and interactivity in a shiny app. Perhaps even some Benford's law analysis ;)

[Stephen-Gates]

State election data via Australia's open data census.

[ivanhanigan]

Great idea, did you know there is a hypothesis in the suicide literature that political regime is correlated with population suicide rates? I am hoping to analysis that relationship myself one day. I'd already sourced some data via the Australian Data Archives: Sharman, Campbell. Australian Election Database - New South Wales House of Assembly Party Election Data. Australian Data Archives. https://www.ada.edu.au/ada/40085 (Retrieved 14 June 2015).

On Sun, Apr 3, 2016 at 8:43 PM, Stephen Gates [email protected] wrote:

State election data via Australia's open data census http://australia.census.okfn.org/dataset/election.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/ropensci/auunconf/issues/20#issuecomment-204940517

[ZhouFang928]

Hi everyone, Good topic. Except for visualization, other possible tasks come into my mind would be

1. predicting election results and detecting potential influencers,
2. analyzing the voter participation across different groups of people (gender, age, income, occupation, ...) and forecasting the future trend I come from Singapore and not very familiar with the Australia data source. Appreciate if anyone could give me some guidance on how to get the access to election data from the links listed above. I am also happy to establish the connection and form a team to work together. Thanks Fang

[benmarwick]

@ZhouFang928 we can download Excel or CSV files from sites such as http://results.aec.gov.au/17496/Website/HouseDownloadsMenu-17496-csv.htm (for the most recent federal election house of reps results), and maybe incorporate them into a package, or write a package that can download them. For voter participation, we might need polling data, I'm not sure how much of that is publicly available.

[cofiem]

I'd recommend including something from openaustralia.org.au. They do excellent work making information from Australian govt Hansard more useful, and have an api and open source.

[dicook]

there is code and data from 2014 available here http://dicook.github.io/stat585/slides/7-data-structures.html#/8, would need to get updated from http://aec.gov.au/Electorates/maps.htm

[dicook]

To look at polling data or results for each electorate it would be good to build a cartogram. Some electorates are very small and would effectively disappear from a regular map

[benmarwick]

Wow, @dicook those slides are great, looks like you've already done a lot of the work for this kind of package!

Here are some more data sources that we might need for this:

Census data:

• Mesh blocks are smallest unit, but only appear to have counts of people and dwellings, this is different from NZ, it seems, where @ellisp has lots of interesting data at the mesh block level. He also has an excellent post here where he talks about how he made his nzelect package, could be useful for getting started with this one.
• We may have to use 'Statistical Area 1' data, which are the smallest spatial unit with interesting data, to answer the kinds of questions @ZhouFang928 mentions. We can get this from https://www.censusdata.abs.gov.au/datapacks/ ('all geographies', or SA1, 'long header', 'basic community profile', and 'Working Population Profile'), which needs a free account to access.

Election data:

• Election results downloads can be obtained http://results.aec.gov.au/17496/Website/Default.htm
• Polling place locations (with lat-long data) are available at http://results.aec.gov.au/17496/Website/GeneralDownloadsMenu-17496-csv.htm or http://www.aec.gov.au/elections/federal_elections/2013/downloads.htm
• Electorate shapefiles can be downloaded from http://www.aec.gov.au/Electorates/gis/gis_datadownload.htm
• If we use these data we need to include a statement like: Copyright statement to be included with AEC data: "This product (XXXX) incorporates data that is: © Commonwealth of Australia (Australian Electoral Commission) 2016"

[tslumley]

A cartogram-like approach from the Grauniad: http://www.theguardian.com/world/datablog/interactive/2013/sep/06/australian-election-results-map

[dicook]

working on it - not too successful yet but hopeful

On Apr 21, 2016, at 2:28 PM, Thomas Lumley [email protected] wrote:

A cartogram-like approach from the Grauniad: http://www.theguardian.com/world/datablog/interactive/2013/sep/06/australian-election-results-map

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

---

Di Cook [email protected]

[dicook]

ok, take a look at this:

i had to split the cities out and work separately on them - darling algorithm explodes with the australian population spread

[tslumley]

Some sort of outline is probably needed, but it will have to be done by hand to account for the distortions.

[tslumley]

I've submitted a pull request with the hexmap data and an example.

[benmarwick]

Some examples of the election data viz. Code for these in https://github.com/ropenscilabs/ausElectR/blob/master/AECdata/tidying_and_testing_AEC_data.R

Proportion of total formal votes for the major parties by electorate (just a sample shown here)

Comparing party and candidate votes of several parties (based on @ellisp's work)

Voting by electorate and state (based on @ellisp's work)

[benmarwick]

I've spent a few moments changing things in the eechidna pacakge to satisfy devtools::check and I'm pleased to report the current result:

I've also made a start on the ropensci checklist and about 2/3 of the way through that. I've contacted the ropensci guys about getting the package into its own repo.

I'd like to propose replacing nat_data with nat_data_cart, because it seems to be the same thing, but with a few more columns. But I haven't been able to find any code that shows how nat_data_cartwas created, it would be good to have a record of that. I can see in the Getting Oz Electorate shapefiles vignette how nat_data was made. Who can update the repo with the code that made nat_data_cart?

I've also added two party preferred election data at the polling place and electorate levels as data objects in the package.

[dicook]

I can handle the nat_data_cart. Keep both data. Need to put dorling code into our package for now. I can do this.

From the little black box...

On 25 Apr 2016, at 4:10 AM, Ben Marwick [email protected] wrote:

I've spent a few moments changing things to satisfy devtools::check and I'm pleased to report the current result:

I've also made a start on the ropensci checklist and about 2/3 of the way through that. I've contacted the ropensci guys about getting the package into its own repo.

I'd like to propose replacing nat_data with nat_data_cart, because it seems to be the same thing, but with a few more columns. But I haven't been able to find any code that shows how nat_data_cartwas created, it would be good to have a record of that. I can see in the Getting Oz Electorate shapefiles vignette how nat_data was made. Who can update the repo with the code that made nat_data_cart?

I've also added two party preferred election data at the polling place and electorate levels as data objects in the package.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[benmarwick]

Great, thanks!

Regarding the dorling code, I already copied over the relevant functions from From https://github.com/chxy/cartogram/ into our cartogram.R.

I know this is a bit dodgy, but I did this because I couldn't get a depends on a github package to work (looking at these instructions).

I think it would be better if we could depend on her github package in case those functions change in the future, any ideas how to do that and still pass the checks?

[dicook]

I dont think it is possible to depend on github package. I will work on it later today. On my way back from spotting koalas on raymond island

From the little black box...

On 25 Apr 2016, at 10:16 AM, Ben Marwick [email protected] wrote:

Great, thanks!

Regarding the dorling code, I already copied over the relevant functions from From https://github.com/chxy/cartogram/ into our cartogram.R.

I did this because I couldn't get a depends on a github package to work (looking at these instructions).

I think it would be better if we could depend on her github package in case those functions change in the future, any ideas how to do that and still pass the checks?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[dicook]

Ben,

Just looking at the code. It looks great! Thank you for all the work that you have done to clean it up!

The code to make the nat_data_cart is in the cartogram,R example code for aec_carto_join_f(). I think you might have worked this out already? Looks like it. It doesn’t write the data out but makes a plots of it.

Couple of things/requests:

(1) Is it possible to make the "Plotting-electorates.Rmd” lower case “p”? I tried to remove/add this last week but it still comes up as uppercase for me. I don’t know how to force the change.

(2) This is a request for Carson: is it possible to get the dorling cartogram version of the electorates with a map background into app as an alternative to the hexagon - to be able to choose either on startup?

(3) I have asked Xiaoyue if she minds us including the cartogram code in our package. The way you have done it is good, its nice ot ask too.

(4) It would be really good to have the polygon and data extraction functions from the electorate spatial data as functions in the package. They are in the vignette right now, but it would be good to have functions also. This might be a request for Heike.

(5) I spell-checked the vignettes.

cheers, Di

On Apr 25, 2016, at 10:16 AM, Ben Marwick [email protected] wrote:

Great, thanks!

Regarding the dorling code, I already copied over the relevant functions from From https://github.com/chxy/cartogram/ into our cartogram.R.

I did this because I couldn't get a depends on a github package to work (looking at these instructions).

I think it would be better if we could depend on her github package in case those functions change in the future, any ideas how to do that and still pass the checks?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

---

Di Cook [email protected]

[benmarwick]

Thanks Di! I was also stymied with renaming that vignette, but then I saw that files can be renamed by editing them in github, and that seems to have worked. Thanks for contacting Xiaoyue, I agree that's the right thing to do.

Good point about the polygon and data extraction functions, it would be great it @heike has the time to move them from the vignette to R/. The internet service here is not good enough for me to download the package needed to run Heike's vignette, so I can't really help with working on that until I get to a better connection in a few weeks.

[benmarwick]

Hi everyone,

I've had some correspondence with @sckott (one of the rOpenSci folks) and he made a separate eechidna repository in the ropenscilabs github org for us.

I've now moved our package there, so please git clone https://github.com/ropenscilabs/eechidna and use that repo for working on the package. This should make it easier to prepare for a CRAN submission and the rOpenSci peer review.