project-ideas
project-ideas copied to clipboard
open-austin
Austin City Council Agenda items on a map
Council and Boards / Commissions agendas are plentiful, long and difficult to parse. How can I know what's going in my area? This tool will parse addresses from various agendas, geocode them, and display them on a google map for an easy overview:
https://github.com/davemcphee/CouncilAgendaMapper
*People who have 30 seconds to spend on finding out if an item of interest is coming up in their area. City staff or Councillors who want alerts if something's going on in their district, etc. *
My UI is trash. What even is "CSS". What's a DIV. marquee tags are deprecated?!? Also, how to insert a POLYGON into MySQL.
Status: working prototype is up: https://highlandneighborhood.com/maps/map.html [Firefox doesn't like my js. Try in Chrome or IE]
The plan is to scrape all City meetings, not just Council but all boards and commissions, and allow users to narrow down results by dates, boards, districts, past dates, etc.
Possibly add a City Councillor voting history on items, board member / commissioner voting records, etc.
Pie in the sky idea: have the city re-write their agendas in a modern open data format, instead of typing it up in MS Word and saving as html. Haha! Yeah.
Great idea! A similar project was started at hack for change: https://github.com/open-austin/council-connect.
Another similar project idea is to parse lobbyist data out of the agendas: https://github.com/open-austin/project-ideas/issues/29.
Pie in the sky idea: have the city re-write their agendas in a modern open data format, instead of typing it up in MS Word and saving as html. Haha! Yeah.
I think the city is in the process of finding a new system for their agendas and such.
@davemcphee @mscarey Are you going to Ship It Weekend? Maybe you can work together?
Yeah I saw that. As I read through the projects a slow horror dawned on me that someone had already done what I'd just built, but they seem fairly different.
Looks like @mscarey's perl scraper no longer works on the current "html" format of the agenda?
Yeah the old scraper was made by Greg. I know it's not running anymore and I don't know if it can still run. I'll be at Ship It.
There is now an dataset on the City's Open Data portal that be be useful for this project or related projects.
https://data.austintexas.gov/Government/Austin-City-Council-Agenda-Items/es7e-878h
So I noticed a small issue on the Austin City Council Agenda Items dataset.
I have the column SireLinkWithDraftBackup linking to the entire agenda instead of to just the backup for that item. For example, Item #6 on the Feb 11th agenda takes you here: https://austin.siretechnologies.com/sirepub/mtgviewer.aspx?meetid=1015&doctype=Agenda instead of here: https://austin.siretechnologies.com/sirepub/agdocs.aspx?doctype=agenda&itemid=54152
It's something I want to fix for the entire dataset.
One of the holdups for me is that I have the .xml files buried in a bunch of different folders, one folder for each meeting.
My computer has python 2.7.8, and my question is: Could anyone help me with a script that pulls out all of the XML files from the various folders that are nested under 'G:\Agenda Office\SIRE Export Files' and drop them into a common folder?
Thanks, John
John, Check out two options. Need to figure out if you need to preserve directory structure incase of same name.
Preserving directory tree. http://stackoverflow.com/questions/25643454/copy-all-files-with-certain-extension-while-maintaining-directory-tree
Not copying directory tree. http://stackoverflow.com/questions/23196512/how-to-copy-all-files-from-a-folder-including-sub-folder-while-not-copying-the?rq=1
-John
On Wed, Mar 30, 2016 at 2:13 PM, johndinning [email protected] wrote:
So I noticed a small issue on the Austin City Council Agenda Items dataset.
I have the column SireLinkWithDraftBackup linking to the entire agenda instead of to just the backup for that item. For example, Item #6 https://github.com/open-austin/project-ideas/issues/6 on the Feb 11th agenda takes you here: https://austin.siretechnologies.com/sirepub/mtgviewer.aspx?meetid=1015&doctype=Agenda instead of here: https://austin.siretechnologies.com/sirepub/agdocs.aspx?doctype=agenda&itemid=54152
It's something I want to fix for the entire dataset.
One of the holdups for me is that I have the .xml files buried in a bunch of different folders, one folder for each meeting.
My computer has python 2.7.8, and my question is: Could anyone help me with a script that pulls out all of the XML files from the various folders that are nested under 'G:\Agenda Office\SIRE Export Files' and drop them into a common folder?
Thanks, John
— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/open-austin/project-ideas/issues/36#issuecomment-203590140
Thank you Woodley, the not copying directory tree was just what I needed!
One of the issues is that Sire actually uses a unique ID for each agenda Item (the ID in this example link: https://austin.siretechnologies.com/sirepub/agdocs.aspx?doctype=agenda&itemid=54910 ) But it's never referenced outside of Sire, making it very difficult to reference a particular item (instead, you need to say Item No 42 on the x/y/x meeting) - and even then, those numbers change as other items are inserted ahead of it during agenda drafting.
Would love to see items referenced by unique ID in whatever you guys end up building :)
Yes, that unique ID is definitely key to creating links to the draft backup materials.
However, once the meeting is past, the complete backup that is on the City Clerk's website is referenced by meeting type, date, and item number. That's what they use in their URL conventions for linking to agenda materials including actions taken at the Council meeting.
For example, Item 24 on the March 3, regular (REG) meeting: http://austintexas.gov/department/city-council/2016/20160303-reg.htm#024
Conversation around the need for City Council Vote Tracking has been kicked around.
One local civic org is starting to collect data on how Councilmembers vote on agenda items that of key interest to them. http://www.atxfriends.org/austin-city-council-voting-record/
Once I get clear of some projects I want to pick this back up in November.
hi @davemcphee! I tried to follow your link to the mapper and got this series of errors. I noticed your code is no longer on that github link.
do you have a current repo? If you have some open issues for these bugs, I'd be happy to link to them and add tags for the help needed.
Hi all - growing pains and moves, it's now here: http://minutes.city/#/city/austin
The project / code was bought by some investors who not surprisingly asked me to take the source it offline.
I've updated the initial app to parse agenda items from a number of US cities, aiming at the 100 largest cities in the US, and expanding to other data sources, such as counties and states. The backend only has a few cities activated right now, without checking I think it's Austin, Philly, Seattle, and Dallas. Maybe. Tons more waiting for approval.
A couple of core features (primarily custom alerts, email alerts and push) are currently disabled while ... "lawyers do lawyer thing?!?s"
If you want to pull ATX agenda items in JSON for your own use, feel free to check out the API calls that the frontend makes. It's somewhat limited in terms of how far back you can go - 5 meetings I think? Don't rely on this being up for ever though ;)
Sorry to be a bit vague.
oh neat! cool to see a project like this take off. Do you think this idea could be closed since it's been effectively graduated to a real thing, and then maybe start a new idea that is sometihng others can contribute to?
Actually, let's not close ideas for that reason. Instead, can this project idea pivot to something that others can work on? @davemcphee