Virtual-env name parameter

[damian-t]

trafficstars

I am running n python projects with potentially conflicting dependencies. Preferably I'd like to use your action to create n virtual-envs on each runner, linked to n caches.

As I understand it, I can use custom_cache_key_element to create the n caches,

 - uses: syphar/[email protected]
   id: cache-virtualenv
   with:
    custom_cache_key_element: project_i

but they all seem to refer to the same virtualenv:

home/runner/.virtualenvs/.venv/

Is this the expected behaviour?

[syphar]

Is this the expected behaviour?

in general yes. custom_cache_key_element was added as a trick to reset the cache, not to handle multiple virtual environments.

Also this action was built for a typical case where you have one project per job, and one virtual environment.

Even when we would allow to define the virtual-env directory, or respect the $VIRTUAL_ENV env (see #156), I see a problem with the PATH.

Right now we add the virtualenv-directory correctly to the PATH, so you can just use the python executables in our steps.

Could you explain how you would see PATH being set for multiple virtual environments?

From what I see you are probably better off building custom caching, though we can discuss any solution here and I'm happy to review any PR adding functionality here.

[damian-t]

We actually have one project per job. But we're building a bunch of Python projects (each with its own GH workflow) on a shared set of self-hosted GH runners. Without dedicated virtual-envs for every project, the python environment on the runner becomes cluttered with mixed dependencies from these projects which is problematic. My idea was to use your action to neatly create dedicated venvs + caching

[syphar]

ah, now I understand. Then, #156 would probably solve the problem for you, and/or probably supporting to define the virtualenv directory with with: virtualenv_dir or something like that.

I'm happy to review any PR solving this, it would be a nice feature to have.

[delicb]

I had the same problem with virtualenv being shared between different projects because they might end up being executed on the same self-hosted runner. custom_virtualenv_dir added in #219 helps here and I can create virtualenv per project. However, there is a potential pitfall - conflicting requirements between different branches in a project. Because of that, I have decided to construct the name of the virtualenv like this: custom_virtualenv_dir: ${{ github.repository }}-${{ hashFiles('requirements.txt', 'requirements-dev.txt') }}.

This seems to work, but I am not happy that I have to explicitly list requirements files, when this action already does an excellent job in tracking them for cache key purposes.

@syphar Would you consider deleting the existing environment before restoring cache content? Maybe make it configurable with an input? Alternatively, maybe part of the environment name can be already calculated cache key (at least requirements file hash), so that no deletion is needed, just new env if there is a cache miss?

I am open to other suggestions as well and might be able to provide a PR.

[syphar]

@delicb I could see arguments for both ways.

@syphar Would you consider deleting the existing environment before restoring cache content? Maybe make it configurable with an input?

while this feels more like a things that should be added to the @actions/cache library (delete target directory before restore), I would accept a PR deleting the target.

Alternatively, maybe part of the environment name can be already calculated cache key (at least requirements file hash), so that no deletion is needed, just new env if there is a cache miss?

This is probably the cleaner solution, you could also just add github.reporistory to the normal cache key and reuse the same key for the environment. Or make the repo part of the path being generated.

Both ways would be breaking changes, but I lean to just doing a major version bump with a nice release description and wouldn't add more settings for this.