boto3 icon indicating copy to clipboard operation
boto3 copied to clipboard

eliminate use of __file__ in boto3.session

Open zsol opened this issue 11 months ago • 4 comments

Python doesn't guarantee that the __file__ global is a string pointing to the name of the Python file hosting the source code of the current module, and certain Python environments (like PyOxidizer) indeed don't set it.

This PR makes boto3.session.Session still work in these environments.

zsol avatar Feb 18 '25 21:02 zsol

Hi @zsol,

Apologies this has been lingering for a bit. I was going back through open PRs to find uses/issues about __file__ and stumbled on this. We recently dropped Python 3.8 support which lets us use some more recent interfaces from importlib.resources.

https://github.com/boto/boto3/pull/4528 and https://github.com/boto/botocore/pull/3464 take a slightly different approach to address this. Do you have any concerns with those or other issues we might be missing for your use case?

nateprewitt avatar May 07 '25 21:05 nateprewitt

Hmm I'm not entirely sure that str(files(...)) will always give you a path that's on disk

zsol avatar May 08 '25 08:05 zsol

Do you happen to have any specific use cases you were seeing variance in? From some quick testing, we weren't able to find any difference between the old and new paths being produced. I'm sure there are some edge cases we're missing though.

Or perhaps if you could point out why str(path(__package__, "data").__enter__()) as a context manager is more resilient to this issue?

nateprewitt avatar May 08 '25 16:05 nateprewitt

Sorry I meant to provide an example but it's proving to be a bit tricky to come up with something short :)

The problem is that files() eventually calls reader.files(), which for example for ZipReader, simply returns a zipfile.Path, without actually unzipping the file. OTOH path() calls into as_file(), which returns a context manager that materializes the file/directory when entering, and cleans it up after exiting.

zsol avatar May 08 '25 17:05 zsol