Eliminate __file__ for importlib.resources

[philipkimmey]

trafficstars

Background

We have an internal tool that we distribute to all developers and we're looking to improve how we distribute that package.

Everything works fine with a .pex-style approach, but eliminating the __file__-oriented assumptions would allow us to use PyOxidizer which we hope will reduce the extra ~400ms of overhead that the pex-based approach introduces. (400ms isn't the end of the world but isn't ideal for a CLI tool our team is using all day.)

Here's a somewhat random grab bag of relevant links:

• https://github.com/boto/boto3/issues/1770
• https://github.com/boto/botocore/pull/1969
• https://github.com/certifi/python-certifi/pull/116
• https://pyoxidizer.readthedocs.io/en/latest/oxidized_importer_behavior_and_compliance.html#no-file

Code

This currently this just contains failing tests that demonstrate the issue. I recognize the test code is pretty rough but couldn't come up with a more clever way to demonstrate the issue (since botocore will usually already be imported in the Python interpreter when this test is run.). (It's possible that the proposed Subinterpreter stuff would make that possible but that's theoretical and I'm not familiar enough to know if it would actually even help.)

[codecov-io]

Codecov Report

Merging #2046 into develop will decrease coverage by 0.09%. The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #2046      +/-   ##
===========================================
- Coverage    93.05%   92.96%   -0.10%     
===========================================
  Files           60       60              
  Lines        10991    10861     -130     
===========================================
- Hits         10228    10097     -131     
- Misses         763      764       +1     

Impacted Files	Coverage Δ
botocore/model.py	97.97% <0.00%> (-0.23%)	:arrow_down:
botocore/docs/bcdoc/restdoc.py	96.26% <0.00%> (-0.17%)	:arrow_down:
botocore/credentials.py	98.40% <0.00%> (-0.13%)	:arrow_down:
botocore/retries/bucket.py	98.48% <0.00%> (-0.09%)	:arrow_down:
botocore/validate.py	98.74% <0.00%> (-0.07%)	:arrow_down:
botocore/compat.py	95.32% <0.00%> (-0.06%)	:arrow_down:
botocore/loaders.py	99.24% <0.00%> (-0.04%)	:arrow_down:
botocore/docs/bcdoc/style.py	95.45% <0.00%> (-0.04%)	:arrow_down:
botocore/client.py	98.56% <0.00%> (-0.04%)	:arrow_down:
botocore/paginate.py	97.80% <0.00%> (-0.03%)	:arrow_down:
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 29c033f...1d5bb85. Read the comment docs.

[mattsb42-aws]

Some historical conversation on this if it helps: https://github.com/boto/botocore/issues/1950

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. Because it has been longer than one year since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment to prevent automatic closure, or if the issue is already closed, please feel free to reopen it.

[mattsb42-aws]

It is still an issue

[tcyrus]

I'm not sure if that search is the best example, as the only code that should cause issues with bundling is the code in the botocore directory. That being said, it's still an issue.

[aBurmeseDev]

Thank you all for chiming in. This was brought up to discuss with the team recently and per team discussion, eliminating file at the base level module might be a breaking change and would have to find a better approach to accomplish it. Closing for now.