juriscraper icon indicating copy to clipboard operation
juriscraper copied to clipboard

CO: Missing dates from Supreme Court opinions

Open sentry[bot] opened this issue 4 years ago • 4 comments

CO: IndexError: list index out of range

Sentry Issue: COURTLISTENER-23R

IndexError: list index out of range
  File "cl/scrapers/management/commands/cl_scrape_opinions.py", line 351, in handle
    self.parse_and_scrape_site(mod, options["full_crawl"])
  File "cl/scrapers/management/commands/cl_scrape_opinions.py", line 320, in parse_and_scrape_site
    site = mod.Site().parse()
  File "juriscraper/AbstractSite.py", line 130, in parse
    self._process_html()
  File "juriscraper/opinions/united_states/state/colo.py", line 27, in _process_html
    date = self.html.xpath(

sentry[bot] avatar Feb 04 '22 22:02 sentry[bot]

Colorado court's staff added an announcement with the upcoming opinions to be released on the next Monday. The text broke the expected pattern to search for the listed opinions' date.

Screenshot from 2022-02-04 17-44-29

Also, the listed file is not an opinon, but a file listing issues which were addressed during oral arguments: https://www.courts.state.co.us/Courts/Supreme_Court/Oral_Arguments/Schedules/November%20orals%20docket.pdf

Since this page is edited by the court's staff at least one a week, I'll check it on Monday to verify for changes in the announcement text and to correct this error in the scraper's code.

satsuki-chan avatar Feb 04 '22 23:02 satsuki-chan

We're getting faster at catching this kind of thing!

mlissner avatar Feb 05 '22 00:02 mlissner

Colorado court's staff changed the announcement's text and updated the link to the opinion released today. The new text follows the expected pattern to search for the opinions' date; but a validation was added to avoid the error reported by Sentry.

Screenshot from 2022-02-07 15-11-28

satsuki-chan avatar Feb 07 '22 21:02 satsuki-chan

The search for the optional citation for the opinions from the Supreme court was added to avoid scraping errors, like the one that occurred with the opinion released on February 7th, 2022.

Screenshot from 2022-02-07 16-01-13

satsuki-chan avatar Feb 07 '22 22:02 satsuki-chan

This was resolved.

flooie avatar Jan 28 '23 14:01 flooie