sowdm

This seems to work: ``` url = "https://home.chicagopolice.org/wp-content/uploads/2022-ISR.zip" storage_options = {'User-Agent': 'Mozilla/5.0'} df = pd.read_csv(url, storage_options=storage_options) ``` UPDATE: Tried this again on work computer and it no longer works

> This seems to work: > > ``` > url = "https://home.chicagopolice.org/wp-content/uploads/2022-ISR.zip" > storage_options = {'User-Agent': 'Mozilla/5.0'} > df = pd.read_csv(url, storage_options=storage_options) > ``` This does not work on my...

https://home.chicagopolice.org/robots.txt results in a blank page. I would assume that this means that nothing is disallowed

This might be worth testing: https://stackoverflow.com/questions/16627227/problem-http-error-403-in-python-3-web-scraping

All data has been successfully added except 2019 data. 2019 data has multiple CSV files in it which pandas does not handle ``` > url = 'https://home.chicagopolice.org/wp-content/uploads/2019-ISR.zip' > storage_options =...

Looks like this package can be used: https://smarie.github.io/python-odsclient/ https://pypi.org/project/odsclient/ Possible examples of using alternatively requests to get data https://github.com/opendatasoft/qgis-ods-plugin/tree/main/Opendatasoft https://github.com/opendatasoft/semantic-bot

This works with odsclient ``` dataset_id = "bpd-traffic-stops" from odsclient import get_whole_dataframe df = get_whole_dataframe(dataset_id, base_url="https://data.burlingtonvt.gov", use_keyring=False) ```

Can generate URLs here for different parameters: https://data.burlingtonvt.gov/explore/dataset/bpd-traffic-stops/api/?rows=3&timezone=&q=call_time%3E2016-08-14T00:31:18&sort=-call_time This works: ``` url = "https://data.burlingtonvt.gov/api/records/1.0/search/?dataset=bpd-traffic-stops&q=call_time%3C2015-08-14T18%3A28%3A07-04%3A00&rows=10000" r = requests.get(url) data = r.json()

Burlington switched to ArcGIS but Long Beach uses open data https://data.longbeach.gov/explore/?q=police&sort=modified

Most recent OpenDataSoft documentation: https://help.opendatasoft.com/apis/ods-explore-v2/