pint icon indicating copy to clipboard operation
pint copied to clipboard

Setting `cache_folder=":auto:"` looks for non-existent file

Open jarq6c opened this issue 3 years ago • 1 comments

It appears that setting cache_folder=":auto:" results in pint looking for a file that used to exist in a completely different python environment.

Environment

Python 3.8.10 build 0.8.0 packaging 21.3 pep517 0.13.0 Pint 0.19.2 pip 22.2.2 pkg_resources 0.0.0 pyparsing 3.0.9 setuptools 65.3.0 tomli 2.0.1 wheel 0.37.1

Steps to duplicate

  1. Create a new python environment (e.g. "env1")
  2. Install pint
  3. Run the example code below
  4. Destroy "env1"
  5. Create a new python environment "env2"
  6. Run the code below in the new environment "env2"
import pint

u = pint.UnitRegistry(cache_folder=":auto:")

Error

Traceback (most recent call last):
  File "$HOME/env2/lib/python3.8/site-packages/pint/registry.py", line 639, in load_definitions
    parsed_files = p.parse(file, is_resource)
  File "$HOME/env2/lib/python3.8/site-packages/pint/parser.py", line 249, in parse
    out.extend(self.parse(path, parsed.is_resource))
  File "$HOME/env2/lib/python3.8/site-packages/pint/parser.py", line 235, in parse
    parsed, content_hash = self._diskcache.load(
  File "$HOME/env2/lib/python3.8/site-packages/pint/_vendor/flexcache.py", line 335, in load
    cache_path = self.cache_path_for(header)
  File "$HOME/env2/lib/python3.8/site-packages/pint/_vendor/flexcache.py", line 297, in cache_path_for
    h = self.cache_stem_for(header)
  File "$HOME/env2/lib/python3.8/site-packages/pint/_vendor/flexcache.py", line 287, in cache_stem_for
    for value in header.for_cache_name():
  File "$HOME/env2/lib/python3.8/site-packages/pint/_vendor/flexcache.py", line 81, in for_cache_name
    for el in self._for_cache_name():
  File "$HOME/env2/lib/python3.8/site-packages/pint/_vendor/flexcache.py", line 182, in _for_cache_name
    yield self.source_path.read_bytes()
  File "/usr/lib/python3.8/pathlib.py", line 1229, in read_bytes
    with self.open(mode='rb') as f:
  File "/usr/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/usr/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '$HOME/env1/lib/python3.8/site-packages/pint/constants_en.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    u = pint.UnitRegistry(cache_folder=":auto:")
  File "$HOME/env2/lib/python3.8/site-packages/pint/registry.py", line 161, in __call__
    obj._after_init()
  File "$HOME/env2/lib/python3.8/site-packages/pint/registry.py", line 2018, in _after_init
    super()._after_init()
  File "$HOME/env2/lib/python3.8/site-packages/pint/registry.py", line 352, in _after_init
    loaded_files = self.load_definitions("default_en.txt", True)
  File "$HOME/env2/lib/python3.8/site-packages/pint/registry.py", line 644, in load_definitions
    raise ValueError("While opening {}\n{}".format(file, msg))
ValueError: While opening default_en.txt
[Errno 2] No such file or directory: '$HOME/env1/lib/python3.8/site-packages/pint/constants_en.txt'

Expected behavior

pint will not look for files in other python environments.

Fix

Set cache_folder="some_directory"

jarq6c avatar Aug 30 '22 15:08 jarq6c

We actually use appdirs (https://github.com/ActiveState/appdirs) under the hood for this. See https://github.com/hgrecco/pint/blob/37a61ede6fbd628c7dc160eb36278cf41c96484c/pint/facets/plain/registry.py#L236

Maybe you can open an issue there, referencing to this.

hgrecco avatar Sep 23 '22 03:09 hgrecco

@hgrecco I'm not sure that this is a bug in appdirs. The cache_folder is correctly determined. IMHO, this is a bug in how the flexparser and flexcache work in 0.22.0.

( I don't yet understand the whole caching logic so I might not have the entire picture correct) In both cases (env1 and env2) the cache_folder will be identical, and so will the hash that is generated. So when the env2 comes around and wants to parse its default_en.txt file, it calculates the hash, and finds a match inside the cache_folder it then loads the .pickle file. This loaded ParsedSource object will now have it's location reflect the path to the old env1 though, so the resolution of the IncludeStatements fails, as e.g. default_locator assumes that the passed location is exists, and the resulting path for e.g. looking up Constants.txt will be invalid.

I'm not sure how much has changed on main compared to 0.22 and I've not yet had a chance to test this. But figured I'd raise this anyhow. Have you considered ensuring that the created hashes - and thus caches - are distinct between different python environments/interpreters?

hassec avatar Nov 09 '23 21:11 hassec