backend icon indicating copy to clipboard operation
backend copied to clipboard

Measurements that don't exist in the API should return the 404 HTTP code

Open jakubd opened this issue 3 years ago • 2 comments

Currently if you do an HTTP GET on an explorer URL that does not exist such as:

https://explorer.ooni.org/measurement/20180111T032529Z_AS4761_2l16dMgzTvsYsM1MWNt0SgFcA7dJi7uUnJececc3Gvx-EF525tU?input=http://www.grindr.com/

you will get an HTTP 200 OK but the body response will say "Measurement not found". A small fix would be to return HTTP 404 in this instance instead. This would make mass-checking explorer URLs for accessibility much less network intensive (on both sides) than getting the full response and checking for a text pattern inside.

jakubd avatar Jun 15 '21 13:06 jakubd

Thanks for suggesting this, it's a good point, however for the purpose of scraping and checking the existence of measurements, I would suggest using the OONI API instead of explorer.

You can query this API endpoints to check the existence of a particular measurement:

https://api.ooni.io/api/v1/measurement_meta?report_id=20180111T032529Z_AS4761_2l16dMgzTvsYsM1MWNt0SgFcA7dJi7uUnJececc3Gvx-EF525tU%5C&input%5C=http://www.grindr.com/

If you get back an empty JSON document ({}), then the measurement does not exist, if instead you get one with populated fields (ex. https://api.ooni.io/api/v1/measurement_meta?report_id=20210615T161141Z_webconnectivity_IE_6830_n1_ahJMrA4sAZPR6hK9&input=https%3A%2F%2Fwww.iucn.org%2F) then it exists.

This API endpoint also doesn't properly set the status code and it should happen here as well.

hellais avatar Jun 15 '21 16:06 hellais

I documented the OONI Explorer request here: https://github.com/ooni/explorer/issues/599

hellais avatar Jun 15 '21 16:06 hellais