opam icon indicating copy to clipboard operation
opam copied to clipboard

`opam update` that are noop should be instantaneous

Open Khady opened this issue 1 year ago • 8 comments

When an opam update sees that nothing changed in a repository the operation should be instantaneous instead of taking 20s. Also opam should check headers like last-modified so that it doesn't actually download the index file if it wasn't changed since the last download.

louis@ahrefs-laptop1-popos1:~$ time opam update

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><><><>
[default] synchronised from https://opam.ocaml.org
Now run 'opam upgrade' to apply any package updates.

real	0m29.922s
user	0m11.714s
sys	0m9.216s
louis@ahrefs-laptop1-popos1:~$ time opam update --debug
00:00.034  CLI                    Parsing CLI version 2.1
00:00.035  GSTATE                 LOAD-GLOBAL-STATE @ /home/louis/.opam
00:00.035  CLIENT                 UPDATE 
00:00.035  RSTATE                 LOAD-REPOSITORY-STATE @ /home/louis/.opam
00:00.300  CACHE(repository)      Loaded /home/louis/.opam/repo/state-3AF31D16.cache in 0.264s
00:00.300  RSTATE                 Cache found
00:00.300  STATE                  LOAD-SWITCH-STATE @ default
00:00.302  CACHE(installed)       Loaded /home/louis/.opam/default/.opam-switch/packages/cache in 0.001s
00:00.803  STATE                  Switch state loaded in 0.503s

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><><><>
00:00.806  PARALLEL               Iterate over 1 task(s) with 3 process(es)
00:00.806  PARALLEL               Starting job 0 (worker 1/3): 0
00:00.806  SYSTEM                 mkdir /tmp/opam-2016727-26b9ac
00:02.395  REPOSITORY             update default from https://opam.ocaml.org
00:02.395  CURL                   pull-repo-update
00:02.395  SYSTEM                 mkdir /tmp/opam-2016727-26b9ac/default.new
00:02.395  SYSTEM                 mkdir /tmp/opam-2016727-f25dcf
00:02.395  PARALLEL               Next task in job 0: /usr/bin/curl --write-out %{http_code}\n --retry 3 --retry-delay 2 --user-agent opam/2.1.4 -L -o /tmp/opam-2016727-f25dcf/index.tar.gz.part -- https://opam.ocaml.org/index.tar.gz
Processing  1/1: [default: http]
00:05.665  PARALLEL               Collected task for job 0 (ret:0)
00:05.672  PARALLEL               Next task in job 0: /usr/bin/tar xfz /tmp/opam-2016727-f25dcf/index.tar.gz -C /tmp/opam-2016727-26b9ac/default.new
00:08.533  PARALLEL               Collected task for job 0 (ret:0)
00:08.533  SYSTEM                 rmdir /tmp/opam-2016727-f25dcf
00:08.790  REPO_BACKEND           diff: /tmp/opam-2016727-26b9ac/{default,default.new}
00:08.790  PARALLEL               Next task in job 0: /usr/bin/diff -ruaN default default.new
00:10.019  PARALLEL               Collected task for job 0 (ret:0)
00:10.019  SYSTEM                 rm /home/louis/.opam/log/patch-2016727-59c89b
00:10.019  SYSTEM                 rmdir /tmp/opam-2016727-26b9ac/default.new
00:15.905  REPOSITORY             update empty, no validation performed
[default] no changes from https://opam.ocaml.org
00:15.905  REPOSITORY             default: applying empty update
00:15.905  PARALLEL               Next task in job 0: /usr/bin/tar cfz /home/louis/.opam/repo/default.tar.gz.tmp -C /tmp/opam-2016727-26b9ac default
Processing  1/1:
00:16.901  PARALLEL               Collected task for job 0 (ret:0)
00:16.901  SYSTEM                 rm /home/louis/.opam/repo/default.tar.gz
00:25.290  PARALLEL               Job 0 finished
00:25.291  FILE(repos-config)     Wrote /home/louis/.opam/repo/repos-config in 0.000s
00:25.291  SYSTEM                 rm /home/louis/.opam/repo/state-3AF31D16.cache
00:25.299  CACHE(repository)      Writing the repository cache to ~/.opam/repo/state-3AF31D16.cache ...
00:26.289  CACHE(repository)      ~/.opam/repo/state-3AF31D16.cache written in 0.990s
00:26.441  SYSTEM                 rmdir /tmp/opam-2016727-26b9ac/default
00:28.076  SYSTEM                 rmdir /tmp/opam-2016727-26b9ac

real	0m28.113s
user	0m11.503s
sys	0m8.367s

In that example I expect the second opam update to take less than 1s

Khady avatar May 19 '23 03:05 Khady