[bug] Conan download_cache is not working on conan 2

[igadmg]

Environment details

• Operating System+version: Windows 10
• Compiler+version:
• Conan version: 2.0
• Python version: 3.11

have setup conan download cache in global.conf but it does not work

My setup core.cache:storage_path=f:/.conan core.download:download_cache=f:/.conan/.cache

but files are redownloaded every conan create rerun

Steps to reproduce

No response

Logs

No response

[memsharded]

Hi @yatagarasu25

Thanks for your report. Could you please clarify which files are being re-downloaded?

For clarification the file download cache has 2 different confs:

conan config list | grep cache
core.cache:storage_path: Absolute path where the packages and database are stored
core.download:download_cache: Define path to a file download cache
tools.files.download:download_cache: Define the cache folder to store downloads from files.download()/get()

• core.download:download_cache: for caching the Conan internal files like conan_package.tgz
• tools.files.download:download_cache: for caching the external/third_party files downloaded by recipes in download() or get() calls.

Is it possible that you are missing the second one?

[igadmg]

Thanks, I completely missed tools.files.download:download_cache It is sort of working now, yet still sometime files are cached sometimes they don't. Looks like that depends if sources are read form yaml or url is passed directly from coananfile.py

[memsharded]

It is sort of working now, yet still sometime files are cached sometimes they don't. Looks like that depends if sources are read form yaml or url is passed directly from coananfile.py

The difference could be if the yaml provide md5/sha1 checksums or not. Only files that provide a checksum are cached in the fileDownloadCache

[jwillikers]

I think it would help if this feature was better documented. If ~ isn't going to be expanded, the documentation should say so and document how to use a cache configured in the user's home directory. Many users are probably going to want to put this somewhere in there home directory, like in ~/.cache. I used the following in global.conf to get this working in Conan 1.60.0.

core.download:download_cache = {{ os.path.join(os.path.expanduser('~'), ".cache", "conan", "download") }}
tools.files.download:download_cache = {{ os.path.join(os.path.expanduser('~'), ".cache", "conan", "download") }}

[memsharded]

This has changed in Conan 2.0, that by default will use a download cache inside the Conan cache. We are testing the new features in ConanCenter, and hopefully it will be publicly launched in 2.0.6.

[artesvisuales326-sudo]

https://github.com/conan-io/docs/issues/3276

[memsharded]

@artesvisuales326-sudo, sorry, you added a link to this same ticket, is this intended?

[duffsterlp]

@memsharded We use the conan download cache extensively in conan 1 and are in the process of migrating to conan 2. I'm confused as to whether the conan download cache is still an available feature in conan 2. I see there are config elements in global.conf pertaining to it - but there is no equivalent documentation to the page from conan 1 (https://docs.conan.io/1/configuration/download_cache.html) Also per your comment here (https://github.com/conan-io/docs/issues/3276#issuecomment-1598919811), it seems like the model has changed in conan 2.

Is the download cache still supported as it exists today in conan 1?

[memsharded]

Hi @duffsterlp

yes, the Conan download cache keeps working as a feature in Conan 2. It is enabled and configured with conf:

core.download:download_cache: To cache Conan artifacts (like conan_package.tgz) downloads
tools.files.download:download_cache: to cache user (recipes download/get) downloads

It is true that there is no separate and clear docs page for it, and now it is kind of partially included in https://docs.conan.io/2/devops/backup_sources/sources_backup.html, and mentioned in the conf page https://docs.conan.io/2/reference/config_files/global_conf.html#core-download-download-cache

It might be worth documenting it in the docs in its own page.

[duffsterlp]

Hi @duffsterlp

yes, the Conan download cache keeps working as a feature in Conan 2. It is enabled and configured with conf:

core.download:download_cache: To cache Conan artifacts (like conan_package.tgz) downloads
tools.files.download:download_cache: to cache user (recipes download/get) downloads

It is true that there is no separate and clear docs page for it, and now it is kind of partially included in https://docs.conan.io/2/devops/backup_sources/sources_backup.html, and mentioned in the conf page https://docs.conan.io/2/reference/config_files/global_conf.html#core-download-download-cache

It might be worth documenting it in the docs in its own page.

Thank you for confirming @memsharded . Should I create a separate issue to get the download cache explicitly documented in conan 2?

[memsharded]

Not necessary, I have tagged this ticket for 2.20, lets try to document it in next release