Bug Report: Codecov comments randomly appearing on years-old PRs

[WardBrian]

Describe the bug

Several PRs, ranging from 3 to 5 years old, have recently recieved codecov bot comments, despite being merged and having their branches deleted from the repo.

https://github.com/stan-dev/cmdstanpy/pull/220#issuecomment-3057090903 https://github.com/stan-dev/cmdstanpy/pull/311#issuecomment-3057163635 https://github.com/stan-dev/cmdstanpy/pull/358#issuecomment-3049676873

Expected behavior

Codecov should not post comments on PRs that are several-years merged.

[skshetry]

I also got codecov comments in multiple PRs that were closed/merged 3-4 years ago.

https://github.com/skshetry/webdav4/pull/94#issuecomment-3059843412 https://github.com/skshetry/webdav4/pull/108#issuecomment-3059802640 https://github.com/skshetry/webdav4/pull/132#issuecomment-3059722065 https://github.com/skshetry/webdav4/pull/100#issuecomment-3059681293 https://github.com/skshetry/webdav4/pull/97#issuecomment-3059803421

[WardBrian]

Fourth one showed up just now: https://github.com/stan-dev/cmdstanpy/pull/219#issuecomment-3063763150

[jameslamb]

We saw this in RAPIDS repos as well.

Here's an example where a new comment was written on July 10, 2025 on a PR that was November 4, 2020: https://github.com/rapidsai/cuml/pull/3112#issuecomment-3056726449

That was the first of these we observed, but there were many across multiple repos. We did not determine a pattern for when these comments appeared.

It led to us fully just turning off commenting:

• https://github.com/rapidsai/cugraph-gnn/pull/256
• https://github.com/rapidsai/nx-cugraph/pull/170
• https://github.com/rapidsai/cudf/pull/19472
• https://github.com/rapidsai/cugraph/pull/5185