aiohttp icon indicating copy to clipboard operation
aiohttp copied to clipboard

Skip benchmarks in ci when running in fork repositories

Open Cycloctane opened this issue 1 month ago • 7 comments

What do these changes do?

Forks cannot use codspeed account, which can make benchmark job in ci fail if aiohttp contributors want to run test workflow on their forks. This pr make unnecessary steps skip if triggered workflow is not in the main repository.

Are there changes in behavior for the user?

aiohttp contributors can now run github action workflows to test changes in their forks without unnecessary errors.

Is it a substantial burden for the maintainers to support this?

Related issue number

Checklist

  • [x] I think the code is well written
  • [ ] Unit tests for the changes exist
  • [ ] Documentation reflects the changes
  • [ ] If you provide code modification, please add yourself to CONTRIBUTORS.txt
    • The format is <Name> <Surname>.
    • Please keep alphabetical order, the file is sorted by names.
  • [x] Add a new news fragment into the CHANGES/ folder

Cycloctane avatar Oct 29 '25 06:10 Cycloctane

CodSpeed Performance Report

Merging #11737 will not alter performance

Comparing Cycloctane:skip-workflows-in-forks (6dc6b3e) with master (72fadb8)

Summary

✅ 59 untouched

codspeed-hq[bot] avatar Oct 29 '25 06:10 codspeed-hq[bot]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 98.74%. Comparing base (72fadb8) to head (6dc6b3e). :warning: Report is 20 commits behind head on master. :white_check_mark: All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #11737   +/-   ##
=======================================
  Coverage   98.74%   98.74%           
=======================================
  Files         127      127           
  Lines       43754    43754           
  Branches     2328     2328           
=======================================
  Hits        43205    43205           
  Misses        389      389           
  Partials      160      160           
Flag Coverage Δ
CI-GHA 98.61% <ø> (ø)
OS-Linux 98.35% <ø> (-0.01%) :arrow_down:
OS-Windows 96.68% <ø> (-0.01%) :arrow_down:
OS-macOS 97.57% <ø> (ø)
Py-3.10.11 97.12% <ø> (-0.01%) :arrow_down:
Py-3.10.19 97.62% <ø> (ø)
Py-3.11.14 97.81% <ø> (-0.01%) :arrow_down:
Py-3.11.9 97.32% <ø> (-0.01%) :arrow_down:
Py-3.12.10 97.42% <ø> (-0.01%) :arrow_down:
Py-3.12.12 97.92% <ø> (-0.01%) :arrow_down:
Py-3.13.9 98.17% <ø> (-0.01%) :arrow_down:
Py-3.14.0 98.12% <ø> (-0.01%) :arrow_down:
Py-3.14.0t 97.19% <ø> (-0.01%) :arrow_down:
Py-pypy3.11.13-7.3.20 97.42% <ø> (-0.02%) :arrow_down:
VM-macos 97.57% <ø> (ø)
VM-ubuntu 98.35% <ø> (-0.01%) :arrow_down:
VM-windows 96.68% <ø> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Oct 29 '25 06:10 codecov[bot]

The preferred solution is storing the plain text token fore Codecov. As for codspeed, I haven't seen it failing. Do you have any examples?

FWIW, the repo checks should be done through repo IDs.

webknjaz avatar Oct 29 '25 07:10 webknjaz

I wonder, though, if benchmarking could still run but just skip uploading the results, conditionally. Could you check?

@Cycloctane have you been able to check if this is possible?

webknjaz avatar Nov 05 '25 12:11 webknjaz

Sorry for the delay. I've checked codspeed docs. Benchmarking can be done separately with pytest-codespeed. But codspeed's github action does not to support skipping uploading.

Codspeed token seems not necessary for public repository. We can safely remove it.

Cycloctane avatar Nov 16 '25 21:11 Cycloctane

pypy test is failing. Because pyo3 has dropped support for pypy3.10.

Cycloctane avatar Nov 16 '25 21:11 Cycloctane

Benchmarking can be done separately with pytest-codespeed. But codspeed's github action does not to support skipping uploading.

Alright, this is blocked on https://github.com/CodSpeedHQ/action/issues/146, then.

Codspeed token seems not necessary for public repository. We can safely remove it.

Sounds good.

webknjaz avatar Nov 21 '25 16:11 webknjaz