knime-geospatial-extension
knime-geospatial-extension copied to clipboard
Add Socrata Data Nodes
Socrata Search: To search the database and get the data list Socrata Data Query: get the data based on domain and ID。
Test workflows can be found here: https://hub.knime.com/center%20for%20geographic%20analysis%20at%20harvard%20university/spaces/Workflow%20Tests/WIP~yP2rfc1sFgXgNhQz/
@UrbanGISer just played around with the nodes and have some suggestions
- Would it make sense/could we move the metadata.domain next to the resource.id column to the beginning of the table since these two info are needed in the Socrata Data Query node
- We should add some progress information to the Socrata Data node. I guess we do not know in advanced how many rows will be returned so we cannot update the percentage progress. However what we can do is we can update the progress message with each iteration of the while True loop and include the current offset so that the user sees that the node is working hard on downloading the data
- We should mention the throttling in the node description and that it can be avoided by creating an application token that can be provided via a new advanced setting
- Does the Socrata Search node only return public data sets? If not can we restrict the result to only show public data sets
- I think it would be also great if we could add an advanced query field (simple text field) to the Socrata Query Node that allows the user to specify an optional simple filter or query which gets simply appended as query parameter to the URL
- e.id colu
Updated accordingly
- [x] Would it make sense/could we move the metadata.domain next to the resource.id column to the beginning of the table since these two info are needed in the Socrata Data Query node
- [x] We should add some progress information to the Socrata Data node. I guess we do not know in advanced how many rows will be returned so we cannot update the percentage progress. However what we can do is we can update the progress message with each iteration of the while True loop and include the current offset so that the user sees that the node is working hard on downloading the data
- [x] We should mention the throttling in the node description and that it can be avoided by creating an application token that can be provided via a new advanced setting
- [x] Does the Socrata Search node only return public data sets? If not can we restrict the result to only show public data sets
- [x] I think it would be also great if we could add an advanced query field (simple text field) to the Socrata Query Node that allows the user to specify an optional simple filter or query which gets simply appended as query parameter to the URL
Merged via https://github.com/spatial-data-lab/knime-geospatial-extension/pull/415