project-ideas
project-ideas copied to clipboard
open-austin
lobbyist data + city council agendas
Concise description:
A visualization of how many lobbyists various companies hired, when they hired them, and what issues they lobbied the Austin city council about (with links to the relevant text, hopefully).
Link (more details/brain dump/alpha)
This might be a chance to salvage some of the work done on the dormant Council Connect project (e.g. the scraper). Background:
Council Connect, a notification service mainly for current (not historical) city council agenda items. Council Connect's Perl scraper, courtesy @tgregoneil.
We could get the lobbyist registration data from data.austintexas.gov and collect city council agenda items from the city website. Then we could search the text of the agenda items for names matching the "client" field in the lobbyist dataset.
Who will use/benefit from this project?
People trying to get up to speed on an ongoing city council issue; anyone who wants an overview of which council issues are the most contentious at a glance.
Project Needs (dev/design/resources)
Scraping (Perl or Python?) Visualization (D3.js?)
Status (in progress, pie-in-the-sky)
pie-in-the-sky
How do you connect lobbyists to agenda items? That seems very difficult to do.
In the lobbyist dataset on the portal, each lobbyist has one line for each of the lobbyist's clients. So really, it's more about matching clients to agenda items. Then we could show how many lobbyists each client had at any time. A naive way to do that would be just to search the text of the agenda items for the name of each client in the dataset, but that still leaves the problem of normalizing the clients' names or making the search fuzzy enough to get a match.
As an example, there are four lobbyists on the data portal registered to "White Lodging Services Corporation". The phrase "WHITE LODGING SERVICES CORP." appears in an item in October 1 meeting agenda (which is a PDF with embedded text). So it would be a matter of making a match between those two variations on the name.
The link to the PDF with the "WHITE LODGING SERVICES CORP." agenda item was http://www.austintexas.gov/edims/document.cfm?id=239704. The steps I took to get to it were to go here: https://austintexas.gov/department/city-council/council-meetings, then follow the link to here: http://www.austintexas.gov/department/city-council/2015/20151001-reg.htm, then click the link to the PDF. I think @tgregoneil's scraper would have captured the agenda item from the PDF if that scraper was still running periodically (which it wasn't).
Matt made a post about the data available recently: https://medium.com/open-austin/city-releases-some-requested-data-about-lobbyists-and-city-council-7c73d5d838ee
Due to the new data that became available, this project is active again, in Open's Austin's #p-lobbyist-campaign Slack channel.