QUERY: No warnings for change to `main` (asv 0.6.2 fails to install package for no discernable reason)

[jwodder]

Attempting to benchmark our project with asv 0.6.2 fails to install the project into the virtualenv, and no message is given as to why. The output from a run of asv run --show-stderr --verbose HEAD^1..HEAD can be found at https://gist.github.com/jwodder/16c46681ed639b6c3dc7d0762d0a98c5. Note that the output ends with:

·· Running '/usr/local/bin/git name-rev --name-only --exclude=remotes/* --no-undefined 0ede8b43ddf3f5810a035dc43b1b50c938b51d91'
   OUTPUT -------->
   dummy
·· Installing 0ede8b43 <dummy> into virtualenv-py3.11
·· Failed to build the project and import the benchmark suite.

(dummy is the name of the branch I was testing on.)

If asv 0.6.1 is used instead, everything is successful.

[HaoZeke]

This is because the default git branch is now main, so earlier in the trace it cannot find the commit to compare to. The solution is to either specify the branch in the configuration or rename the default.

[jwodder]

specify the branch in the configuration

Do you mean via the branches config option? The documentation for that says it only applies to asv run ALL and asv run NEW, neither of which I'm running. (The docs also still say that the Git default is "master".)

[jwodder]

@HaoZeke I'm still having this problem with asv 0.6.3. I tried the following:

• On a non-master branch:
  • Set branches to ["master"] and commit
  • Create another commit so that branches will be set in both HEAD and HEAD^1
  • Run asv run HEAD^1..HEAD

However, this fails with "Failed to build the project and import the benchmark suite." and no details as to what actually went wrong. Looking at the logs generated with --show-stderr --verbose, it looks like the installation was successful. The logs can be found at https://gist.github.com/jwodder/e69f53f662cc9cbc1a5fdf2d89ae5649.

[dcherian]

I'm seeing a similarly mysterious error for https://github.com/xarray-contrib/flox

Running with --verbose shows the real cause: asv run --verbose --python=same --bench cohorts

   OUTPUT -------->
   Traceback (most recent call last):
     File "/Users/deepak/miniforge3/envs/flox-tests/lib/python3.11/site-packages/asv/benchmark.py", line 28, in <module>
   from asv_runner.discovery import _discover

     File "/Users/deepak/miniforge3/envs/flox-tests/lib/python3.11/site-packages/asv_runner/discovery.py", line 8, in <module>
     from ._aux import update_sys_path
   File "/Users/deepak/miniforge3/envs/flox-tests/lib/python3.11/site-packages/asv_runner/_aux.py", line 7, in <module>
     from .benchmarks._maxrss import set_cpu_affinity

     File "/Users/deepak/miniforge3/envs/flox-tests/lib/python3.11/site-packages/asv_runner/benchmarks/__init__.py", line 51, in <module>
       asv_modules = [
                   ^
   File "/Users/deepak/miniforge3/envs/flox-tests/lib/python3.11/site-packages/asv_runner/benchmarks/__init__.py", line 54, in <listcomp>
     if dist.metadata["Name"].startswith("asv_bench")

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   AttributeError: 'NoneType' object has no attribute 'startswith'

I don't really understand why there is a None result though.

[saraedum]

I get an error that seems similar to the one you are seeing @jwodder. Adding --verbose does not add any information. I put some print statements into asv to understand what's going on. In my case, the problem is that the default install_command uses wheel_file but wheel_file is only set here https://github.com/airspeed-velocity/asv/blob/8877eff343807e8bc594690dec08bc0da2caa401/asv/environment.py#L813, the code path is not entered and then wheel_file substitution in the install_command fails.

To debug, it helps a lot to add a print(last_err) in this finally block: https://github.com/airspeed-velocity/asv/blob/8877eff343807e8bc594690dec08bc0da2caa401/asv/benchmarks.py#L211

[saraedum]

In my particular case, there was an extra wheel from a dependency being collected. Setting "build_command": ["python setup.py build", "python -mpip wheel --no-deps --no-build-isolation --no-index -w {build_cache_dir} {build_dir}"], in my asv.conf.json fixed the problem.

[dcherian]

@saraedum that works for me too!

[HaoZeke]

I'm not really sure there's much to do here, these are valid fixes, but all at the user/downstream project level related to installations which are due to a default being changed. Closing for now.

[Jacob-Stevens-Haas]

I'm not really sure there's much to do here, these are valid fixes, but all at the user/downstream project level related to installations which are due to a default being changed. Closing for now.

I don't know about this one. (a) IIUC, the project-level configuration that was suggested is to disable letting asv handle the build dependencies. Would that work if the build dependencies to change between commits? (b) The title is about a command failing with no error despite --verbose, and solving it requires adding diagnostic print statements to asv code. That's still an issue, as I ran into today (though not sure for the same reason, i.e. a dependency's wheel popping up in the cache dir).

A few more documentation OBTWs that would have made troubleshooting easier: I love that the developer docs include the process of asv run, but maybe more detail? And it's not clear exactly which parts of which version of asv.conf.json is used if it changes between commits?