trio icon indicating copy to clipboard operation
trio copied to clipboard

Switch to `uv pip compile`

Open CoolCat467 opened this issue 1 year ago • 6 comments

This pull request follows up on #2956 and is attempting to switch the continuous integration system to use astral.sh' new tool uv. This particular pull request is a subset of #2957 and is in charge of only replacing pip-compile.

This seems nontrivial, can we break this into 2 prs?

1. replace pip-compile

2. replace pip install

Personally I see (1) as much more valuable-- (2) doesn't change much. Personally I'm more excited by uv's ability to get the lowest compatible version so we can add a new CI run than by it replacing pip install.

Originally posted by @A5rocks in https://github.com/python-trio/trio/issues/2957#issuecomment-1947713570

CoolCat467 avatar Feb 16 '24 03:02 CoolCat467

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.63%. Comparing base (17b3644) to head (96446a5). Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2958   +/-   ##
=======================================
  Coverage   99.63%   99.63%           
=======================================
  Files         120      120           
  Lines       17830    17830           
  Branches     3204     3204           
=======================================
  Hits        17765    17765           
  Misses         46       46           
  Partials       19       19           

codecov[bot] avatar Feb 16 '24 03:02 codecov[bot]

uv seems to support keeping keeping things like implementation_name now: https://github.com/astral-sh/uv/pull/4503

I think once this uses that flag, this is good for me

A5rocks avatar Jun 26 '24 00:06 A5rocks

So, added --no-strip-markers, but since there hasn't been a proper release with these changes yet, using a git commit dependency for uv at the moment, which works, but CI fails because full cargo isn't installed on the windows or alpine runners.

I think we just need to wait for a proper release with these changes to come out, but otherwise this pull request should be almost done!

CoolCat467 avatar Jun 26 '24 01:06 CoolCat467

I think we just need to wait for a proper release with these changes to come out, but otherwise this pull request should be almost done!

Heads up we plan to cut a release today.

charliermarsh avatar Jun 26 '24 12:06 charliermarsh

Ok so it looks like everything is working except for an issue that happens when importing cryptography on the 3.13 runner, with this error: E ImportError: /opt/hostedtoolcache/Python/3.13.0-beta.2/x64/lib/python3.13/site-packages/_cffi_backend.cpython-313-x86_64-linux-gnu.so: undefined symbol: _PyErr_WriteUnraisableMsg

I don't think this is caused by uv, probably an issue with cryptography. Version that 3.13 runner used to have this result: cryptography-42.0.8

CoolCat467 avatar Jun 27 '24 06:06 CoolCat467

I'm surprised pulling in master didn't work. I manually edited the test requirements to use a pre-release of cffi (IIRC? Or maybe cryptography) to fix that.

A5rocks avatar Jun 27 '24 06:06 A5rocks

Weird SSL issue on MacOS 3.12 runner:

  ________________________ test_ssl_client_basics[tls13] _________________________
    + Exception Group Traceback (most recent call last):
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/runner.py", line 341, in from_call
    |     result: Optional[TResult] = func()
    |                                 ^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/runner.py", line 241, in <lambda>
    |     lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    |     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    |     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    |     raise exception.with_traceback(exception.__traceback__)
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    |     teardown.throw(exception)  # type: ignore[union-attr]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_call
    |     yield from thread_exception_runtest_hook()
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/threadexception.py", line 63, in thread_exception_runtest_hook
    |     yield
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    |     teardown.throw(exception)  # type: ignore[union-attr]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_call
    |     yield from unraisable_exception_runtest_hook()
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 65, in unraisable_exception_runtest_hook
    |     yield
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    |     teardown.throw(exception)  # type: ignore[union-attr]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    |     yield from self._runtest_for(item, "call")
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/logging.py", line 833, in _runtest_for
    |     yield
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    |     teardown.throw(exception)  # type: ignore[union-attr]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/capture.py", line 878, in pytest_runtest_call
    |     return (yield)
    |             ^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    |     teardown.throw(exception)  # type: ignore[union-attr]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    |     return (yield)
    |             ^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    |     res = hook_impl.function(*args)
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/runner.py", line 183, in pytest_runtest_call
    |     raise e
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/runner.py", line 173, in pytest_runtest_call
    |     item.runtest()
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/python.py", line 1632, in runtest
    |     self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    |     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    |     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    |     raise exception.with_traceback(exception.__traceback__)
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    |     res = hook_impl.function(*args)
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/_pytest/python.py", line 162, in pytest_pyfunc_call
    |     result = testfunction(**testargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/testing/_trio_test.py", line 44, in wrapper
    |     return _core.run(
    |            ^^^^^^^^^^
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_core/_run.py", line 2305, in run
    |     raise runner.main_task_outcome.error
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_tests/test_ssl.py", line 436, in test_ssl_client_basics
    |     async with ssl_echo_server(client_ctx) as s:
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 217, in __aexit__
    |     await anext(self.gen)
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_tests/test_ssl.py", line 190, in ssl_echo_server
    |     async with ssl_echo_server_raw(**kwargs) as sock:
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 217, in __aexit__
    |     await anext(self.gen)
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_tests/test_ssl.py", line 172, in ssl_echo_server_raw
    |     async with trio.open_nursery() as nursery:
    |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_core/_run.py", line 959, in __aexit__
    |     raise combined_error_from_nursery
    | ExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
    +-+---------------- 1 ----------------
      | Traceback (most recent call last):
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_threads.py", line 437, in to_thread_run_sync
      |     return msg_from_thread.unwrap()
      |            ^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/outcome/_impl.py", line 213, in unwrap
      |     raise captured_error
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_threads.py", line 363, in do_release_then_return_result
      |     return result.unwrap()
      |            ^^^^^^^^^^^^^^^
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/outcome/_impl.py", line 213, in unwrap
      |     raise captured_error
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_threads.py", line 382, in worker_fn
      |     ret = context.run(sync_fn, *args)
      |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/trio/_tests/test_ssl.py", line 134, in ssl_echo_serve_sync
      |     wrapped.unwrap()
      |   File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 1296, in unwrap
      |     s = self._sslobj.shutdown()
      |         ^^^^^^^^^^^^^^^^^^^^^^^
      | ssl.SSLWantWriteError: The operation did not complete (write) (_ssl.c:2685)
      +------------------------------------
  ----------------------------- Captured stdout call -----------------------------
  ssl_echo_serve_sync got unexpected error: The operation did not complete (write) (_ssl.c:2685)

Pretty sure it isn't related to this pull request's changes though, going to try to get CI to run again. Might be an instance of test flakiness.

CoolCat467 avatar Jul 08 '24 00:07 CoolCat467