pyjuliapkg icon indicating copy to clipboard operation
pyjuliapkg copied to clipboard

discover dependencies in pyproject.toml files

Open cjdoris opened this issue 1 year ago • 0 comments

This is sort of a prerequisite to #16 - that issue aims to specify dependencies in pyproject.toml for packages, instead of needing to add a juliapkg.json file in the package.

This issue suggests changing the behaviour of juliapkg to look for pyproject.toml files in the current directory and parent directories. If one is found then:

  • Read dependencies from the [tools.juliapkg] table there, in addition to all the other places (the venv root dir and all installed packages).
  • Make this directory the place where the Julia environment gets put (instead of the venv root dir).
  • Make this file be the default place where juliapkg.add() etc put their dependencies.

This doesn't have any effect on packaging, which will need an additional mechanism to get the dependencies into the built package. This suggestion simply makes it easier to implicitly work in projects nested within a virtual environment.

To be clear, if I do

python -m venv ~/.venv
source ~/.venv/bin/activate
pip install juliacall
python -c "import juliacall"

then juliapkg will get dependencies from ~/.venv/juliapkg.json and any packages installed in the venv. Dependencies are installed into ~/.venv/julia_env.

But if I then do

mkdir foo
cd foo
echo 'name = "my-project"' >pyproject.toml
python -c "import juliacall"

then juliapkg will get dependencies from ~/.venv/foo/pyproject.toml in addition to ~/.venv/juliapkg.json and any packages installed in the venv. These dependencies are installed into ~/.venv/foo/julia_env. Doing juliapkg.add() from this directory will add to pyproject.toml.

Some questions:

  • Is this a breaking change, as presented?
  • Should it be opt-in behaviour?
  • Should we ignore pyproject.toml if it doesn't have a [tools.juliapkg] table, and therefore fall back on installing into ~/.venv/julia_env in the above example? This avoids creating lots of extra Julia projects for Python project that don't specify extra Julia dependencies. This would solve the previous question - you'd be opting in by creating the [tools.juliapkg] table. We could make a helper function for this juliapkg.init_project() which creates the table and sets the project.
  • What if there are pyproject.toml files in multiple parent directories? Do we stop at the first or get dependencies from them all?

cjdoris avatar Jul 17 '24 12:07 cjdoris