Josh
Josh
Part of https://github.com/Police-Data-Accessibility-Project/PDAP-Scrapers/issues/80 When a Scraper is run and an Extraction is submitted successfully, we should be dropping a URL / path into the [data_intake database](https://www.dolthub.com/repositories/pdap/data-intake).
Let's look at the scrapers we have and the fields they scrape, to see what we can learn from them. Each scraper has a fields.txt.
## The task: - [ ] Represent these requirements in the scrapers readme or template as appropriate - [ ] Represent them by creating an example scraper that meets the...
> If we could have a subpage to test the scrapers on, that'd be great. Basically two separate pages, both having a pdf with the same name, but different data.
Currently: https://airtable.com/app473MWXVJVaD7Es/tblx8XaKnFTphWNQM/viwIlzibc9W3IEfHW?blocks=hide This is our own database, without row limits! ## Rationale This will become the foundation of most of our early work. ## Requirements - [ ] A mechanism...
## Quick search Allows people to quickly search for Data Sources by: what kind of data it is and where agency is that the data is about (by name, location,...
Currently: https://airtable.com/shrUAtA8qYasEaepI/tblx8XaKnFTphWNQM In general we're making this: # Brief This is a big complicated search interface which help people find what they need in our [Data Sources db](https://airtable.com/app473MWXVJVaD7Es/tblx8XaKnFTphWNQM/viw9mmOR0fw8HFOje?blocks=hide). Most of...
Follows https://github.com/Police-Data-Accessibility-Project/planning/issues/145 We discussed using GitHub Actions to do this. We'll need to have secrets: https://docs.github.com/en/rest/actions/secrets At first, this repo will just have a few CSVs which periodically update. That's...
## Sort _We may decide we don't need this._ - We should have a nice default sort - Nice to have: We should show "good" data sources on an unfiltered...