docker icon indicating copy to clipboard operation
docker copied to clipboard
verified publisher icon pelias

Failed to download openaddresses data for Belgium, Luxembourg & The Netherlands

Open narzero opened this issue 3 years ago • 5 comments

Describe the bug When running pelias download oa I get the following error

pelias download oa
info: [openaddresses-download] Attempting to download selected data files: nl/countrywide.csv,be/bru/bosa-region-brussels-nl.csv,be/bru/bosa-region-brussels-fr.csv,be/vlg/bosa-region-flanders-nl.csv,be/wal/bosa-region-wallonia-fr.csv,lu/countrywide.csv
info: [openaddresses-download] Downloading nl/countrywide.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/nl/countrywide.zip
warn: [openaddresses-download] failed to download https://results.openaddresses.io/latest/run/nl/countrywide.zip: Error: cURL request failed, HTTP status: 301, exit code: 47
info: [openaddresses-download] Downloading be/bru/bosa-region-brussels-nl.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/be/bru/bosa-region-brussels-nl.zip
warn: [openaddresses-download] failed to download https://results.openaddresses.io/latest/run/be/bru/bosa-region-brussels-nl.zip: Error: cURL request failed, HTTP status: 301, exit code: 47
info: [openaddresses-download] Downloading be/bru/bosa-region-brussels-fr.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/be/bru/bosa-region-brussels-fr.zip
warn: [openaddresses-download] failed to download https://results.openaddresses.io/latest/run/be/bru/bosa-region-brussels-fr.zip: Error: cURL request failed, HTTP status: 301, exit code: 47
info: [openaddresses-download] Downloading be/vlg/bosa-region-flanders-nl.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/be/vlg/bosa-region-flanders-nl.zip
warn: [openaddresses-download] failed to download https://results.openaddresses.io/latest/run/be/vlg/bosa-region-flanders-nl.zip: Error: cURL request failed, HTTP status: 301, exit code: 47
info: [openaddresses-download] Downloading be/wal/bosa-region-wallonia-fr.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/be/wal/bosa-region-wallonia-fr.zip
warn: [openaddresses-download] failed to download https://results.openaddresses.io/latest/run/be/wal/bosa-region-wallonia-fr.zip: Error: cURL request failed, HTTP status: 301, exit code: 47
info: [openaddresses-download] Downloading lu/countrywide.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/lu/countrywide.zip
warn: [openaddresses-download] failed to download https://results.openaddresses.io/latest/run/lu/countrywide.zip: Error: cURL request failed, HTTP status: 301, exit code: 47
info: [openaddresses-download] All done!

Steps to Reproduce

  1. Set the following data for openaddress in pelias.json:
"openaddresses":{
   "datapath":"/data/openaddresses",
   "files":[
      "nl/countrywide.csv",
      "be/bru/bosa-region-brussels-nl.csv",
      "be/bru/bosa-region-brussels-fr.csv",
      "be/vlg/bosa-region-flanders-nl.csv",
      "be/wal/bosa-region-wallonia-fr.csv",
      "lu/countrywide.csv"
   ]
}
  1. Run pelias download oa

Expected behavior I expect the needed data to be downloaded from openadresses.

Environment (please complete the following information):

  • OS: Ubuntu 18.04 LTS
  • Docker versions: Docker version 19.03.13, build 4484c46d9d, docker-compose version 1.17.1

narzero avatar Dec 13 '22 14:12 narzero

There is a big red header here https://results.openaddresses.io which makes me think the time to migrate from results.openaddresses.io -> batch.openaddresses.io has come.

Screenshot 2022-12-13 at 16 37 41

missinglink avatar Dec 13 '22 15:12 missinglink

I believe that the older files are still available, it might be that their server had connectivity issues 🤷

missinglink avatar Dec 13 '22 15:12 missinglink

I believe that the older files are still available, it might be that their server had connectivity issues 🤷

It’s been like this since yesterday. And the endpoint is still giving the same error. Maybe it really is time to switch to the new endpoints?

narzero avatar Dec 13 '22 17:12 narzero

Accidentally closed with the comment, reopening...

narzero avatar Dec 13 '22 17:12 narzero

FYI this is being handled in https://github.com/pelias/openaddresses/pull/513

missinglink avatar Jan 23 '23 14:01 missinglink