juriscraper icon indicating copy to clipboard operation
juriscraper copied to clipboard

Fill `fla` gap

Open grossir opened this issue 1 year ago • 1 comments

Related to #929

Between September 06, 2019 and December 18, 2019 we have 0 documents. We are missing 35 documents

To solve this, a dynamic backscraper will be implemented. Also taking the change to change the code to OpinionSiteLinear, deleting some boilerplate

grossir avatar Mar 20 '24 16:03 grossir

Command to fill the gap

docker exec -it cl-django python /opt/courtlistener/manage.py cl_back_scrape_opinions --courts juriscraper.opinions.united_states.state.fla --backscrape-start=09/05/2019 --backscrape-end=12/19/2019

grossir avatar May 02 '24 21:05 grossir

So, I was expecting 35 documents but the backscraper only got 30.

As an example of what's happening, I ran the backscraper on my local env, and got 5 opinions for 2019-11-27, while we only have 3 for those dates after the backscrape: 1,2,3 Comparing with the downloaded ones on my env

 2019-11-27 | 58334727b75985e272472b0a2023f9e9ff85be19 | Byron McGraw v. State of Florida
 2019-11-27 | 6293ff185b969d49bdcc4b8ac97655f4b473ecd5 | In re: Amendments to Florida Rules of Appellate Procedure 9.120 and 9.210 – Rehearing Order
 2019-11-27 | eea385b7ce00efce6eb7ede30e1d73fb49ee6fc7 | In re: Amendments to the Florida Rules for Qualified and Court-Appointed Parenting Coordinators – Notice of Correction
 2019-11-27 | 4a5b74ed2bcdab37dbe1883e779fd5900e29a137 | In re: Amendments to the Florida Rules of Juvenile Procedure - 2019 Fast-Track Report
 2019-11-27 | ed4da94c62e8a80daa1eefa83e9c89471da6b25b | In re: Certification of Need for Additional Judges

I get these missing hashes, which indeed exist. So, they seem to be the same document re-published, which is why the backscraper is not picking it up. I get the same for the other missing documents

https://www.courtlistener.com/api/rest/v3/opinions/?sha1=4a5b74ed2bcdab37dbe1883e779fd5900e29a137 https://www.courtlistener.com/api/rest/v3/opinions/?sha1=58334727b75985e272472b0a2023f9e9ff85be19

Having understood the apparent issue, I am closing this as completed

grossir avatar May 29 '24 00:05 grossir