Daniel Thorn
Daniel Thorn
right now we depend on consumers of the "Glean probe-scraper" github action to report unexpected failures, such as in https://github.com/mozilla/probe-scraper/issues/574 We need to add monitoring so that can be aware...
https://github.com/mozilla/bedrock/actions/runs/4598182706/jobs/8121878736 https://github.com/mozilla/glean/actions/runs/4609412250/jobs/8146505104?pr=2441 gsutil is failing to download objects that fail with 404 exceptions: ``` Error: Command ['gsutil', '-q', '-m', 'rsync', '-r', 'gs://probe-scraper-prod-artifacts/glean/', '/tmp/tmpy4y4leon/output/glean'] returned non-zero exit status 1: NotFoundException: 404...
revision in tree integration/mozilla-inbound isn't available outside of probe-scraper's cache: ``` Retreiving Buildhub results for channel nightly 4645 revisions found ... Downloading files for revision number 494/4645 - revision: 46fe2115d46a5bb40523b8466341d8f9a26e1bdf,...
when a git repository is specified multiple times in `repositories.yaml` it is cloned separately for each instance. Given the size of some of these repos, and the likelihood of repeatedly...
this code fails when there are commits where a file was deleted: https://github.com/mozilla/probe-scraper/blob/c71f5dd8b0f4a98a9c2de83b38f22ff4e0b40205/probe_scraper/scrapers/git_scraper.py#L128 https://github.com/mozilla/probe-scraper/blob/c71f5dd8b0f4a98a9c2de83b38f22ff4e0b40205/probe_scraper/scrapers/git_scraper.py#L84-L85 the current workaround requires listing every commit where the file was deleted: https://github.com/mozilla/probe-scraper/blob/c71f5dd8b0f4a98a9c2de83b38f22ff4e0b40205/probe_scraper/scrapers/git_scraper.py#L35-L49 https://github.com/mozilla/probe-scraper/blob/c71f5dd8b0f4a98a9c2de83b38f22ff4e0b40205/probe_scraper/scrapers/git_scraper.py#L103 https://github.com/mozilla/probe-scraper/blob/c71f5dd8b0f4a98a9c2de83b38f22ff4e0b40205/probe_scraper/scrapers/git_scraper.py#L123-L124 That...