offenesdresden
[Köln] Data
http://www.koeln.de/apps/parken/
https://twitter.com/tursics/status/611411296664354816
We got a notice from netcologne that we shouldn't scrape the website but instead use alternative sources. They sent me the following link: https://offenedaten-koeln.de/dataset/parkhausbelegung As I see I already commented the data source given in the link above.
As they clearly disapproved scraping the website we should change the data source asap. I see that the link @florianschmidt1994 and the link I posted seem to be consistent. @AugustQu I'd be really happy if you could change the website scraper to read the Data from these sources. The Data should be easy to parse as it is already JSON. Also I suggest to use http://www.koeln.de/apps/parken/json/current only for the static data (maybe just transform it once into our geojson) and http://www.stadt-koeln.de/externe-dienste/open-data/parking.php for the dynamic data. The identifiers should be used to map the data.
I created a converter for the static data in https://github.com/offenesdresden/ParkAPI/commit/1948efbe164ace920d03e188f8aa1295ddaae8ed and a prototype in https://github.com/offenesdresden/ParkAPI/commit/bd3069448bad0df2380ff85893b78d0ffe020310.
The only current problem are the opening hours from which we want to create our status. They have different formattings and different keys.
OK, I will look into that topic.
The data seems to be extracted and displayed correctly. I will look into the alternate source.
@AugustQu thanks, but I already did that yesterday as it is quite time critical, so there's no need to do that twice. I just wasn't sure if I get it done soon enought.
As said above we got a notice from netcologne that they dislike scraping their website. So I want to stop using their website as soon as possible.
I fixed the parking lot state in 9a1468eb77df0536648f3b7296d4ba35b823652d. I will update the server to use this version. But I set Kölns active_support to false since the check if a parking lot is open is far to imprecise to be a first class city. It currently marks a city as open if the original source says that it is permanently open. All other options are handled as unknown. Since our apps handle unknown similar to nodata probably closed parking lots will be shown as good options. Also even if we would change the behaviour to that those would be shown as closed (?) we had many false positives (I know that there are other cities with an unknown state but those parking lots are usually really open while we have many specific examples of lots here that are mostly closed but will always be shown as open). Since I can't answer these problems but have to change the implementation to the API I have to set it as not actively supported city (aka. beta and no default city).
Solution idea:
The most parking lots have three different fields: open, open_time and close_time. The first one seems to be always available and the other two are sometimes, and even if they're available they can be empty.
As for the data we should have in the aux field of the geojson. This could be a table (dict) that contains an open and a close time for each week day (special events not considered) and the parser would check when it runs and calculate the correct state from this.
This data needs to be parsed from https://www.koeln.de/apps/parken/json/current (see tools/geojson_koeln.py). The problem is that open is mostly German prose (so we might need some kind of simple language processing or really fancy parser semantics) and that the other two mentioned fields are not reliably available. Another option would be that someone does this by hand but then the generation of the geojson file is cumbersome.
A further problem for this case are some parking lots that seem to be event based without any further information. So we can't automatically check if they're open or not so they should till go as unknown. One could go with heuristics (aka if the lot is completely empty its probably closed, if it has some vehicles on it it is probably open, etc.).
Also what would make things easier if the apps would handle the unknown option better in regards to how to show them. how to sort them, etc. but this is content for another issue and not part of this repo.