WD_CACHE_DISABLED=True is not respected

[larsrinn]

Is your feature request related to a problem? Please describe.

Running this

import os
import time

os.environ["WD_CACHE_DISABLE"] = "True"

import wetterdienst

logs

Wetterdienst cache is enabled

Describe the solution you'd like

Log: Wetterdienst cache is disabled

And also: I don't think logging a warning in both directions is appropriate. For us, this clutters our Sentry account with issues that aren't really issues. Wouldn't a log-level of info be sufficient?

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context

Output of wetterdienst.info()

Wetterdienst - Open weather data for humans-------------------------------------------
version:	 0.42.1
authors:	 Benjamin Gutzmann <[email protected]>, Andreas Motl <[email protected]>
documentation:	 https://wetterdienst.readthedocs.io/
repository:	 https://github.com/earthobservations/wetterdienst
cache_dir:	 /Users/xxx/Library/Caches/wetterdienst

[larsrinn]

Also, it might be of interest how I noticed that: My MacOS complained about my disk being full. When looking into it, it turned out the wetterdienst-cache-folder had a size of more than 500 GB.

I don't want to use the cache. But if you want to, I think a configurable maximum cache size with a reasonable default would be useful.

[gutzbenj]

Dear @larsrinn ,

I added a first fix for the caching configuration although we will probably apply further changes regarding the current way we handle concurrency with settings.

As for the maximum cache size we are relying on the features implemented in fsspec and there it is currently not possible to define a maximum cache size unfortunately. I will however try to see if we can make our own implementation for this.

[amotl]

Dear Lars,

apologies that you experienced troubles with Wetterdienst cache storage.

Wetterdienst cache is enabled

At https://github.com/earthobservations/wetterdienst/issues/827#issuecomment-1355835985:

That log message was misleading, it was fixed with GH-828.

At https://github.com/earthobservations/wetterdienst/issues/827#issuecomment-1355822602:

Using both Settings.cache_disable = True, or environ['WD_CACHE_DISABLE'] = 'True', to turn off caching, worked perfectly well for me. I was able to confirm that no directory has been created at either /Users/amo/Library/Caches/wetterdienst (macOS) or /root/.cache/wetterdienst (Linux/Docker), after running the program.

---

My MacOS complained about my disk being full. When looking into it, it turned out the wetterdienst-cache-folder had a size of more than 500 GB.

I am very sorry about that. As @gutzbenj outlined, there is apparently no option to limit the size of the cache. Regarding the WD_CACHE_DISABLE setting, there may have been hiccups in the past. Can you confirm it works with recent versions of Wetterdienst?

With kind regards, Andreas.

[gutzbenj]

After the latest settings refactoring, this now works as expected. Closing for now but we may come back if cache size can be limited in a future release of fsspec.