sigstore-python icon indicating copy to clipboard operation
sigstore-python copied to clipboard

Resolve ResouceWarning and DeprecationWarning messages for 3.11

Open djhenderson opened this issue 3 years ago • 5 comments

Resolve ResouceWarning and DeprecationWarning messages for 3.11

This example presents ResourceWarning and DeprecationWarning messages that appear to be from sigstore sources (and other source, which are out of scope).

> py -3.11 -E -W default -m sigstore verify python-3.11.0rc2-amd64.exe
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\certifi\core.py:36: DeprecationWarning: path is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  _CACERT_CTX = get_path("certifi", "cacert.pem")
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_internal\rekor\client.py:36: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  _DEFAULT_REKOR_ROOT_PUBKEY = resources.read_binary("sigstore._store", "rekor.pub")
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_internal\rekor\client.py:37: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  _STAGING_REKOR_ROOT_PUBKEY = resources.read_binary(
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_internal\rekor\client.py:41: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  _DEFAULT_REKOR_CTFE_PUBKEY = resources.read_binary("sigstore._store", "ctfe.pub")
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_internal\rekor\client.py:42: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  _STAGING_REKOR_CTFE_PUBKEY = resources.read_binary(
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_verify.py:63: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  DEFAULT_FULCIO_ROOT_CERT = resources.read_binary("sigstore._store", "fulcio.crt.pem")
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_verify.py:64: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  DEFAULT_FULCIO_INTERMEDIATE_CERT = resources.read_binary(
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_verify.py:68: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  STAGING_FULCIO_ROOT_CERT = resources.read_binary(
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_verify.py:71: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
  STAGING_FULCIO_INTERMEDIATE_CERT = resources.read_binary(
OK: python-3.11.0rc2-amd64.exe
C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sigstore\_cli.py:453: ResourceWarning: unclosed <ssl.SSLSocket fd=648, family=2, type=1, proto=0, laddr=('192.168.0.14', 2122), raddr=('104.155.154.165', 443)>
  for file, inputs in input_map.items():
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Version

> sysinfo
Computer network name:    mars
Machine type:             AMD64
Processor type:           Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
Platform type:            Windows-10-10.0.19043-SP0
Operating system:         Windows
Operating system release: 10
Operating system version: 10.0.19043
> py -3.11 -V
Python 3.11.0rc2

djhenderson avatar Sep 12 '22 16:09 djhenderson

Thanks for this report as well. The socket leak looks like the same one in #211, but I'll use this issue to handle the separate deprecation warnings for importlib.resources.

woodruffw avatar Sep 12 '22 16:09 woodruffw

Looks like the importlib.resources documentation on docs.python.org needs to be updated for 3.11: read_binary and read_text are not currently marked as deprecated there.

woodruffw avatar Sep 12 '22 16:09 woodruffw

I think based on https://github.com/python/cpython/issues/93610 they were marked as deprecated, but you might want to follow up there.

di avatar Sep 12 '22 19:09 di

I'll tackle this today.

woodruffw avatar Oct 04 '22 16:10 woodruffw

Hmm, I don't think there's much we can do about these DeprecationWarnings, at least not until we move up to Python 3.9 as our minimum Python version: the APIs that replace and deprecate read_text and read_binary don't appear until then.

Removing the "bug" label for now, since these are not replaceable in our currently supported versions.

woodruffw avatar Oct 04 '22 16:10 woodruffw

I do maintain importlib_resources to enable access to files() (and other new behaviors) on Python 2.7+. Of course, you may have reasons not to adopt new dependencies, but if you can adopt a new dependency, you can rely on importlib_resources; python_version < "3.9" and then in your code conditionally import importlib_resources or importlib.resources depending on Python version. I'd recommend that over awaiting the retirement of Python 3.8, but YMMV.

jaraco avatar Nov 26 '22 18:11 jaraco

Thanks @jaraco! Yeah, I'm in favor of moving away from these APIs if there's a high-quality shim we can use, which importlib_resources certainly looks like :slightly_smiling_face:

woodruffw avatar Nov 26 '22 20:11 woodruffw

#341 fixes this. Thanks again for the tip @jaraco!

woodruffw avatar Dec 13 '22 18:12 woodruffw