azure-sdk-for-python icon indicating copy to clipboard operation
azure-sdk-for-python copied to clipboard

`replace_dev_reqs` function utilized within `dispatch_tox.py` should re-use `PREBUILT_WHEEL_DIR`

Open scbedd opened this issue 1 year ago • 1 comments

azure-storage-extensions is being added as a dev_req to azure-storage-blob in another PR from @jalauzon-msft .

As a necessary prestep before actually invoking our parallel tox environments, we replace the relative deps in each dev_requirements.txt file for the packages we will be testing. This is absolutely necessary, as otherwise python executables will run into conflicts accessing those relative directories.

When adding azure-storage-extensions as a dev_requirement, now it suddenly needs to be built. This is a native package and can't really be built without the necessary requirements being installed (which we rely on cibuildwheel for in the build job).

We need to honor the prebuilt wheel directory when replacing all the dev_requirements. If a wheel exists that matches the relative dep package + version, we should use the prebuilt wheel that matches.

scbedd avatar Feb 09 '24 22:02 scbedd

This issue can be reproduced / fix tested in this Storage feature branch: https://github.com/Azure/azure-sdk-for-python/tree/feature/storage-content-validation/

~~Currently the dev_req looks like this but feel free to adjust as needed (I think it may need to be adjusted to a local path) https://github.com/Azure/azure-sdk-for-python/blob/feature/storage-content-validation/sdk/storage/azure-storage-blob/dev_requirements.txt~~

jalauzon-msft avatar Feb 28 '24 21:02 jalauzon-msft

Took a look at this with @jalauzon-msft, these are the recommended steps:

# add argument to  replace_dev_reqs 

  # evaluate the line to see if its a relative dep
  # use discover_prebuilt_wheel(path)
#<from root>
python -m pip install -r eng/ci_tools.txt
$env:TF_BUILD="true"
python scripts/devops_tasks/tox_harness.py azure-storage-blob --toxenvs "whl" --prebuilt_whl_dir = <path-to-downloaded-folder>

scbedd avatar Mar 21 '24 18:03 scbedd