GLOBAL.plugins resolution behaves differently when using PANTS_SOURCE

[cburroughs]

To use pants.backend.url_handlers.s3 one is advised to add botocore to GLOBAL.plugins:

https://github.com/pantsbuild/pants/blob/7b2dcad1b9649b9005ba9683d1a0e09e29ef5127/src/python/pants/backend/url_handlers/s3/register.py#L33

However, how GLOBAL.plugins works is different when PANTS_SOURCE is used. With this sample pants.toml

[GLOBAL]
pants_version = "2.21.0"
plugins=[
    "botocore~=1.34.138"
]

backend_packages = [
  "pants.backend.url_handlers.s3"
]

$ pants list ::
16:42:42.49 [INFO] waiting for pantsd to start...
16:42:42.89 [INFO] pantsd started
16:42:43.48 [INFO] Starting: Resolving plugins: botocore~=1.34.138
16:42:50.01 [INFO] Completed: Resolving plugins: botocore~=1.34.138
16:42:50.09 [INFO] Initializing scheduler...
16:42:50.41 [INFO] Scheduler initialized.
16:42:50.44 [INFO] Starting: Resolving plugins: botocore~=1.34.138
16:42:53.79 [INFO] Completed: Resolving plugins: botocore~=1.34.138
16:42:53.87 [WARN] No targets were matched in goal `list`.

But using git checkout origin/2.21.x

$ PANTS_SOURCE=~/src/o/alt-pants/ pants list ::
Bootstrapping pants_deps with requirements:
  /home/ecsb/src/o/alt-pants/3rdparty/python/requirements.txt
...
Exception message: 1 Exception encountered:

ProcessExecutionFailure: Process 'Resolving plugins: botocore~=1.34.138' failed with exit code 1.
stdout:

stderr:
pid 3002 -> /home/ecsb/.cache/pants/named_caches/pex_root/venvs/6c991f58350b82e9e7f918782549c384bb7d25bc/108a3ddc84230ab282ea6312e06cb68f51008ce5/bin/python -sE /home/ecsb/.cache/pants/named_caches/pex_root/venvs/6c991f58350b82e9e7f918782549c384bb7d25bc/108a3ddc84230ab282ea6312e06cb68f51008ce5/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /home/ecsb/.cache/pants/named_caches/pex_root/pip/24.0/pip_cache --log /tmp/pants-sandbox-3gIkVh/.tmp/pex-pip-log.srnq5ieg/pip.log download --dest /home/ecsb/.cache/pants/named_caches/pex_root/downloads/resolver_download.xpsoxecn/home.ecsb..cache.pants.pants_dev_deps.d210acf08c6584ef7163e4775332b6ffeeeaaa83.venv.bin.python3.9 --constraint __constraints.txt botocore~=1.34.138 --index-url https://pypi.org/simple/ --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Cannot install botocore==1.34.138, botocore==1.34.139, botocore==1.34.140, botocore==1.34.141, botocore==1.34.142 and botocore==1.34.143 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
 
 The conflict is caused by:
     botocore 1.34.143 depends on urllib3<1.27 and >=1.25.4; python_version < "3.10"
     botocore 1.34.142 depends on urllib3<1.27 and >=1.25.4; python_version < "3.10"
     botocore 1.34.141 depends on urllib3<1.27 and >=1.25.4; python_version < "3.10"
     botocore 1.34.140 depends on urllib3<1.27 and >=1.25.4; python_version < "3.10"
     botocore 1.34.139 depends on urllib3<1.27 and >=1.25.4; python_version < "3.10"
     botocore 1.34.138 depends on urllib3<1.27 and >=1.25.4; python_version < "3.10"
     The user requested (constraint) urllib3==2.2.2
 
 To fix this you could try to:
 1. loosen the range of package versions you've specified
 2. remove package versions to allow pip attempt to solve the dependency conflict




Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

The Lockfile at https://github.com/pantsbuild/pants/blob/7b2dcad1b9649b9005ba9683d1a0e09e29ef5127/3rdparty/python/user_reqs.lock#L2124

has urllib3 at 2.2.2, so the PANTS_SOURCE may be "more correct" even though the vanilla invocation appears to work.

[kaos]

Yea, running off of pants sources constrains your code to Py 3.9, which may result in different resolutions for dependencies. That's my hunch at least.