dune icon indicating copy to clipboard operation
dune copied to clipboard

[pkg] fetch dependencies without building the project

Open maiste opened this issue 1 year ago • 4 comments

Context

In the current behaviour, unless you have the exact target, it is impossible to download the dependencies without building the project. A nice feature would be a command that fetch the project dependencies in the same way you would with dune build. This is important in the context of Dockerfile, where you can save some computation time by caching this stage.

Solution

We can implement a dune pkg fetch command that triggers the fetch rule for the entire project.

maiste avatar Sep 23 '24 13:09 maiste

What about just introducing an alias @pkg-fetch? This is the more standard mechanism for collecting targets in dune.

rgrinberg avatar Sep 23 '24 15:09 rgrinberg

Yes, definitely! It makes more sense. Another proposition was dune build @deps.

The original question was if there was a way to mimic the behaviour of opam install --deps-only --with-test . in the context of Dune Package Management. It means that it should fetch the target and build them too. However, I don't know if we have the possibility to execute the test part?

maiste avatar Sep 23 '24 15:09 maiste

Thanks for opening the issue @maiste 🙏🏼 To clarify, we have 2 use-cases here.

  1. Writing a Dockerfile where we can stage building dependencies separate from building the entire project, and cache that based off the dune-project (or eventually the dune.lock):
FROM ubuntu

RUN curl https://get.ocaml.org | bash 

# Adds project + deps
ADD dune-project .
RUN dune pkg lock

# Fetch and build dependencies only if the above changes
RUN dune build @deps 

# Adds rest of the sources
ADD . .
RUN dune build
  1. A single command to fetch all external / downloadable resources for a project, to let you work without connectivity:
$ git clone github.com/my/project project
$ cd project
$ dune fetch
$ # now i can go offline
$ dune build 

leostera avatar Sep 23 '24 18:09 leostera

The original question was if there was a way to mimic the behaviour of opam install --deps-only --with-test . in the context of Dune Package Management. It means that it should fetch the target and build them too. However, I don't know if we have the possibility to execute the test part?

We already build the test deps. So dune runtest should work.

rgrinberg avatar Sep 23 '24 19:09 rgrinberg

I have work on part of this in #11046 where it introduces an alias to install and build packages. However, as stated here, we would like to have another alias to just fetch the dependencies but without building them. It is currently unclear to me what would be the proper way to address this second part. Indeed, the call to Fetch.fetch in source.ml seems to also build the dependency.

@rgrinberg would you have a suggestion on where I could introduce the alias?

Does it mean that I have to split the function in two parts so we prevent the building?

maiste avatar Oct 28 '24 14:10 maiste

Is this issue fixed now?

rgrinberg avatar Nov 24 '24 13:11 rgrinberg

Yes, it was fixed by #11046.

maiste avatar Nov 25 '24 07:11 maiste