giessdenkiez-de
giessdenkiez-de copied to clipboard
Improve README.md
The README needs (a lot) more information.
- [ ] How to setup?
- [ ] How to develop?
- [ ] How to deploy?
- [ ] How to contribute?
- [ ] How to file issues (platform users)?
- [ ] Where are the lambda functions?
- [ ] How to prepare the data?
- [ ] Where is the DB?
- [ ] Where does the rain data come from?
- [ ] What does the harvester do?
- [ ] Where are the shields/badges and emojis?
@joergreichert Please ask questions here. So we can write down what forks need and include it into our README.md
Thanks for the offer:
- you can see the diff here: https://github.com/CodeforLeipzig/tsb-trees-frontend/commit/60ba83e5b0ce2c0ab20d860477353bfcf816b17a
- for some unknown reason cannot open your https://tsb-trees.s3.eu-central-1.amazonaws.com/trees.csv.gz with 7-zip under Windows (so not sure what is contained) - I've just replaced it completely with geojson file generated from https://opendata.leipzig.de/dataset/strassenbaumkataster Shape file
- you are using standalter where the Leipzig data is using year of planting (which is more stable I think)
- Leipzig data contains only Art (although they are using sometimes strange values like Stadtbaum) where you have Art and Gattung, so I cannot fill the second attribute
- I'm not sure where the radolan_days are coming from, is it already contained in your trees.csv.gz ?
- On the other hand there are seem to be addition map layers that may contain the weather data?
- for the algorithmic: you take the radolan data (e.g. from here https://www.dwd.de/DE/leistungen/radolan/radolan_info/home_freie_radolan_kartendaten.html ), sum it up for the last 60 days and categorize it with the usually required amount of water to keep a tree healthy? (might be adapted when someone waters the tree manually)
- I've checked in my readonly MAPBOX-TOKEN for convenience, as I understand, Mapbox is only required for this isometric view?
- I have not found the place in code, where to change the map center point
- I was told that auth0.com might not be that great in sense of protection of data privacy
- for some unknown reason cannot open your https://tsb-trees.s3.eu-central-1.amazonaws.com/trees.csv.gz with 7-zip under Windows (so not sure what is contained) - I've just replaced it completely with geojson file generated from https://opendata.leipzig.de/dataset/strassenbaumkataster Shape file
I think @sebastian-meier can answer that question
- you are using standalter where the Leipzig data is using year of planting (which is more stable I think)
- Leipzig data contains only Art (although they are using sometimes strange values like Stadtbaum) where you have Art and Gattung, so I cannot fill the second attribute
Can't say anything about the dataset. I also refer this to @sebastian-meier and @fdnklg
- I'm not sure where the radolan_days are coming from, is it already contained in your trees.csv.gz ?
The radolan data is aggregated from the DWD Radar data. You can find the script that does a daily update of this here github.com/technologiestiftung/dwd-radolan-tree-harvester/. It runs on an AWS Fargate every day and writes directly into our Postgres DB. Also running on AWS RDS.
- On the other hand there are seem to be addition map layers that may contain the weather data?
- for the algorithmic: you take the radolan data (e.g. from here https://www.dwd.de/DE/leistungen/radolan/radolan_info/home_freie_radolan_kartendaten.html ), sum it up for the last 60 days and categorize it with the usually required amount of water to keep a tree healthy? (might be adapted when someone waters the tree manually)
Also two questions for @fdnklg or @sebastian-meier
- I've checked in my readonly MAPBOX-TOKEN for convenience, as I understand, Mapbox is only required for this isometric view?
Mapbox is required to get the map + tiles working. We are using deck.gl on top of the base map drawn with react-mal-gl. More Docs on this can be found here deck.gl | docs | using-with-react
- I have not found the place in code, where to change the map center point.
The initial viewport is set in src/state/Store.js
- I was told that auth0.com might not be that great in sense of protection of data privacy
Don't know never heard of any issues with them. The user data is currently stored in the EU in an DB at Auth0.com running our own auth system is not a good idea since so we need to trust in someone.
We choose Auth0 because we can protect several APIs working together with it. A good example to get started with the Auth0 React code is this one auth0.com/docs/quickstart/spa/react
There are two more components that are not really mentioned in the README
- The API that connects to the PG DB running on Vercel/now (formerly Zeit) github.com/technologiestiftung/tsb-tree-api-now-express
- The Auth0 Management API connector running on Vercel as well github.com/technologiestiftung/tsb-trees-api-user-management which allows us to retrieve usernames and allow the user to nuke their accounts.
Both of these only work by sending tokens created through the Auth0 API.
Hey @joergreichert,
CSV The https://tsb-trees.s3.eu-central-1.amazonaws.com/trees.csv.gz is a gzipped csv. But, for convenience, there is also an unzipped one: https://tsb-trees.s3.eu-central-1.amazonaws.com/trees.csv
Standalter In our database backend we are also using the date when it was planted. One of our biggest obstacles was getting the data transferred to the user as small as possible, and when you have 600k+ items, every bit counts. So instead of sending 4 digits for year, we now send a maximum of 3 mostly 2 digits.
Additional Metadata Additional attributes are not super important to the application, you simply need to modify the react-component for the details card.
radolan_days The CSV only holds the radolan_sum, which is the water for the last 30 days. radolan_days are stored in the database and transferred when a user clicks on a tree. See also the trees-api.
weather data All the weather data is being collected, optimised and applied to the trees every morning in this python script: https://github.com/technologiestiftung/dwd-radolan-tree-harvester/ (./harvester/). This script also generates two geojsons for the weather grid. A light version with only the sum of the last 30 days. And a big one, which contains more data points for each day for the last 30 days.
mapbox If you are only interested in the desktop version, mapbox is only used for background tiles and georeferencing. But if you also want the mobile version, then the harvester needs an additonal api-key with write-access. Because the harvester uploads the latest weather data to mapbox and then uses mapbox's api to generate new vector-tiles.
General note The https://github.com/technologiestiftung/dwd-radolan-tree-harvester/ and the PostgreSQL Database are extremely important for the overall application, as the script generates a lot of static files every morning, which are required for proper display of the trees. Please let me know if you have any questions about the harvester.
Thanks for answering my questions so far. Will need to take a closer look at the harvester.
Hi joerg,
I'd like to get in touch with you. Unfortunately I did not found a way to send you a personal message. Which is the best way?
Hi joerg,
I'd like to get in touch with you. Unfortunately I did not found a way to send you a personal message. Which is the best way?
I've just added you at Twitter, so I maybe a private message there.
@joergreichert @tipuraneo I created a rough diagram what the relations are. @sebastian-meier I added this diagram to the docs folder. It can be edited with https://www.diagrams.net would be cool if you could extend the part of the harvester, or let me know in a little more detail what is happenning in the harvester and S3.
Hallo @joergreichert ,
hier ist Thorsten von Stiftung "Ecken wecken". Im Rahmen von Wir im Quartier unterstützen wir das Lösungsteam Baumfreundschaft – Wir gießen im Quartier.
Julia Zimmermann von der Technologiestiftung Berlin hatte mich auf Deine Aktivitäten zu Gieß den Kiez Leipzig aufmerksam gemacht.
Es wäre prima, wenn wir uns darüber mal telefonisch austauschen könnten. Würd mich freuen, wenn Du mich unter 0177 - 26 79 408 anrufen könntest.
Viele Grüße Thorsten
Stiftung "Ecken wecken" Thorsten Mehnert, Vorstand Markranstädter Straße 29 b (Zugang über graues Stahltor) D - 04229 Leipzig
Internet: https://stiftung-ecken-wecken.de E-Mail: [email protected] Tel: +49 - 177 - 26 79 408
Du möchtest nicht nur träumen, sondern Deine Ideen auch umsetzen? Diese Lösungsteams arbeiten bereits an ihren Ideen fürs Quartier.