chaoss
Core:detect_github_repo_move_core error on repos that have been deleted
Error handling needed for repositories that have been deleted
Example: https://github.com/anaconda-platform/transformers-feedstock
Exception('ERROR: Repo has moved, and there is no redirection! 404 returned, not 301. Resetting Collection!')
Traceback (most recent call last):
File "/opt/venv/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
File "/opt/venv/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
File "/augur/augur/tasks/github/detect_move/tasks.py", line 27, in detect_github_repo_move_core
ping_github_for_repo_move(session, key_auth, repo, logger)
File "/augur/augur/tasks/github/detect_move/core.py", line 118, in ping_github_for_repo_move
raise Exception("ERROR: Repo has moved, and there is no redirection! 404 returned, not 301. Resetting Collection!")
Exception: ERROR: Repo has moved, and there is no redirection! 404 returned, not 301. Resetting Collection!
When you say "repositories that have been deleted", I think you are talking about repositories that no longer exist and have redirection.
The design question, which is not trivial, is whether or not we want the Augur instance to preserve that history, and if so, how would that be signaled to an end user?
The description field of the repo table is changed to this or something like it when this occurs:
During our check for this repo on 2025-06-09, a 404 error was returned. The repository does not appear to have moved. Instead, it appears to be deleted
Question: How do we define or ensure that 404's resulting from GitHub being down are not "permanent". CHECK a standard API OK endpoint after getting 404?
There is a function call to update_repos_with_dict that is suppose to update the repo URL in the repo table if its changed .. this is the part we aren't sure is happening.
https://github.com/chaoss/augur/pull/3248#discussion_r2437279863
I am pretty sure this error is not happening anymore but the repo_gits are not geting updated
it still seems like its in the code, but also this isnt actually an error, its just basically reporting that the repo has moved and there was no redirect (and therefore it will be ignored in future) using an exception. which is i think what #3248 is fixing
This exception seems to be put here intentionally to stop other facade tasks from running after this one on the same repo if this task detects a repo is gone (404) and marks it as IGNORE.
Is there a more graceful way to handle that?
https://dev.to/onticdani/how-to-cleanly-stop-celery-tasks-on-exceptions-1fa0 could be helpful here
ok so the key issue here is that this task tries to stop collection by throwing an exception, but because its defined as part of a celery Group (not a Chain) of tasks, tasks are treated as independent.
its added to a list (which is later coerced into a group by default) here: https://github.com/chaoss/augur/blob/3a97f18b250b0f3ff6f5938b603be14f02c36bd8/augur/tasks/start_tasks.py#L166-L168
@sgoggins should this preliminary task (aka the move detection task) run in a chain before the rest of the tasks so that it has the ability to throw an exception (or better yet a celery Retry exception) so that it can retry the tasks after updating the repo_git without throwing an exception to the user and creating a visible task failure when such fialure is the happy path?
turns out it is running in a chain as far as i can tell. that makes things easy
https://github.com/chaoss/augur/blob/3a97f18b250b0f3ff6f5938b603be14f02c36bd8/augur/tasks/util/collection_util.py#L536-L539