gh-135401: Test AWS-LC as a cryptography library in CI

[WillChilds-Klein]

Notes

Currently in draft status to test the new workflow. Building AWS-LC, linking the interpreter to it dynamically, and running the ssl tests all appear to work fine. I'm still working on remaining issues in test_hmac.

TODO

• @WillChilds-Klein to fix test_hmac failures in AWS-LC PR 2484

---

• Issue: gh-135401

[picnixz]

Can you cherry-pick 8f4a0eb739713b5baf28aefa9d561873a70e25ef and make a separate PR please? TiA.

[picnixz]

Ok, the failure is because HMAC-SHA3 isn't supported in AWS-LC. I don't know if the ValueError is actually on my side or fired from OpenSSL and I'm just converting the message, but improving that message would be nice.

[WillChilds-Klein]

Looks like it's coming from python. This ValueError will be fixed once PR 2484 has been merged.

[picnixz]

Ok so it fell back to the default error message (i.e. there was no reason we could extract)

[picnixz]

Ah yes the error is due to multissltests. We only use tags but the script could be extended to support exact commits maybe?

[WillChilds-Klein]

Ah yes the error is due to multissltests. We only use tags but the script could be extended to support exact commits maybe?

I suppose it could, but that would require taking a test container dependency on git. I followed the pattern set by OpenSSL -- download a source zip for a specific release. AWS-LC should release v1.55 ~soon containing the HMAC-SHA3 implementation that this CR needs.

[WillChilds-Klein]

There's a lot of duplication between the build-ubuntu-ssltests-openssl and build-ubuntu-ssltests-awslc jobs, can we use a matrix to combine them?

Something like https://github.com/hugovk/cpython/commit/a3f2ba9eb0c9bd1927d9a34faed98234afe88c70 -> https://github.com/hugovk/cpython/actions/runs/16121165851

I think that makes a lot of sense. @AA-Turner previously suggested that I leave use OpenSSL's established CI definition patterns for this PR, then follow up with another PR to refactor CI definitions.

@hugovk @AA-Turner -- Would you prefer that I incorporate the CI refactor into this PR or a fast-follow-up PR?

[picnixz]

I think it's better to make it in a separate one. It'll be easier to revert if bad things happen.

[zware]

I think we've quibbled enough on this; I'm merging it :). I'm re-running the new job one last time just to be sure, but setting automerge anyway. Since it's non-blocking anyway, we can fix up anything else in followups.

[bedevere-bot]

:warning::warning::warning: Buildbot failure :warning::warning::warning:

Hi! The buildbot ARM Raspbian Linux Asan 3.x (no tier) has failed when building commit db47f4d844acf2b6e52e44f7f3d5f7566b1e402c.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1811/builds/16) and take a look at the build logs.
4. Check if the failure is related to this commit (db47f4d844acf2b6e52e44f7f3d5f7566b1e402c) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1811/builds/16

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to 'db47f4d844acf2b6e52e44f7f3d5f7566b1e402c'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at db47f4d844a gh-135401: Test AWS-LC as a cryptography library in CI (GH-135402)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module

In file included from ./Include/internal/pycore_dict.h:11,
                 from Objects/typeobject.c:7:
In function ‘Py_DECREF_MORTAL’,
    inlined from ‘PyStackRef_XCLOSE’ at ./Include/internal/pycore_stackref.h:730:9,
    inlined from ‘_PyThreadState_PopCStackRef’ at ./Include/internal/pycore_stackref.h:810:5,
    inlined from ‘vectorcall_maybe’ at Objects/typeobject.c:3103:9:
./Include/internal/pycore_object.h:481:8: warning: array subscript 0 is outside array bounds of ‘PyObject[0]’ {aka ‘struct _object[]’} [-Warray-bounds]
  481 |     if (--op->ob_refcnt == 0) {
      |        ^

Timeout (0:05:00)!
Thread 0x0000007f969ef100 [Thread-2] (most recent call first):
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/subprocess.py", line 2019 in _try_wait
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/subprocess.py", line 2047 in _wait
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/subprocess.py", line 1278 in wait
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 194 in _run_process
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 299 in run_tmp_files
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 363 in _runtest
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 403 in run
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/threading.py", line 1074 in _bootstrap_inner
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/threading.py", line 1036 in _bootstrap

Thread 0x0000007f971ff100 [Thread-1] (most recent call first):
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/subprocess.py", line 2019 in _try_wait
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/subprocess.py", line 2047 in _wait
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/subprocess.py", line 1278 in wait
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 194 in _run_process
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 299 in run_tmp_files
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 363 in _runtest
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 403 in run
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/threading.py", line 1074 in _bootstrap_inner
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/threading.py", line 1036 in _bootstrap

Thread 0x0000007fa2f756c0 [python] (most recent call first):
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/logger.py", line 42 in log
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 553 in _get_result
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/run_workers.py", line 610 in run
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/main.py", line 455 in _run_tests_mp
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/main.py", line 561 in _run_tests
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/main.py", line 598 in run_tests
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/main.py", line 767 in main
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/libregrtest/main.py", line 775 in main
  File "/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/Lib/test/__main__.py", line 2 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
make: *** [Makefile:2460: buildbottest] Error 1

Cannot open file '/home/buildbot/buildarea/3.x.pablogsal-rasp.asan/build/test-results.xml' for upload

[hugovk]

@WillChilds-Klein Thank you very much for this!

[WillChilds-Klein]

@hugovk -- No problem at all. And many thanks to all the maintainers for your support and guidance here.

By my count there are a few follow-ups to address:

1. @AA-Turner's request for delayed consolidation of matrix/env var configuration in .github/workflows/build.yml
2. @hugovk's request to de-duplicate ssl-related build/test configurations in .github/workflows/build.yml
3. @picnixz's concern around code structure specific to Tools/ssl/multissltests.py
4. @hugovk's request to colorize test output
5. @zware's open question around 2.7 support

I'll cut a PR for follow-ups 1. and 2. this coming week. I'll also work on a PR for 4. unless someone beats me to it.

For 3. -- @picnixz you indicated that you want to handle the refactor. Please feel free to delegate that to me if you like, else I'm happy to review your refactor.

For 5. -- Again, this is a maintainer question around 2.x support, but if you all want me to remove 2.7 from aforementioned comment in one of my upcoming PRs, I'm happy to do so.

[picnixz]

Please feel free to delegate that to me if you like, else I'm happy to review your refactor.

Feel free to pick up this task / hold it. The refactorization is not really pressing as it's only for extending the script. For now, we can keep it.

[zware]

5. @zware's open question around 2.7 support

I would support updating the compatibility comment to just "all currently-maintained versions" rather than listing any particular ones and modernizing accordingly.