Daniel Holth

Pickle is the conda objects like SubdirData which do not translate back into the exact server-sent json. It would be interesting to pickle the unmodified json. Of course that would...

@mariusvniekerk we will benchmark the whole thing, and we have zstandard available because of `.conda` https://dholth.github.io/conda-benchmarks/#conda_install.TimeInstall.time_explicit_install?conda-package-handling=1.9.0&conda-package-handling=2.0.0a2&p-threads=1&p-threads=5

FYI I talked to an internal team that also needs the old cache filenames, will update to keep the filenames the same.

This feature is available in conda 23.3.1 with the `"--experimental=jlap"` flag. It is the most impressive after the third cache expiration - on an empty cache, we fetch repodata.json; the...

@wolfv @ocefpaf @beckermr @mariusvniekerk Maybe you are not all still voting members, but how do you feel about this CEP now that there is a complete implementation?

@baszalmstra @jaimergp ping

On my machine, memray shows a peak of about 420MB for `repodata = json.load(open("/Users/dholth/miniconda3/pkgs/cache/09cdf8bf.json"))` which is the conda-forge noarch repodata.json. Some users do have more compute than bandwidth.

In Python, it might be possible to have a sqlite-backed repodata.json that returns proxy objects for `"packages"` and `"packages.conda"`, and apply typical `"patch": [{"op": "add", "path": "/packages/pre_commit-3.4.0-hd3eb1b0_1.tar.bz2" ...` without loading...

@travishathaway It's designed so that the checksum doesn't match at the end. We don't specify the server's json formatting. But we keep track of the on-disk versus putative "logically equivalent...

If we look at https://gist.github.com/dholth/cc76ce07f1c6ff099f440fc901bea35b, which are the 'op' and 'path' properties of individual json patch elements, it shows that almost all of the patch operations in the current https://conda.anaconda.org/conda-forge/linux-64/repodata.jlap...