Support for pipenv

[pmatos]

Can we add support for pipenv to elpy? https://docs.pipenv.org/

I think the most urgent thing is to support virtualenvs as created by pipenv.

[jorgenschaefer]

What would support for virtualenvs as created by pipenv entail?

[pmatos]

Good question, leave it to me to do some research.

[aparkerlue]

Thanks for bringing this up.

Elpy seems to work fine with Pipenv virtualenvs if you set virtualenvwrapper's WORKON_HOME to Pipenv's virtualenv directory, which I think is an excellent location:

export WORKON_HOME=$HOME/.local/share/virtualenvs

With the environment variable set, ~/.local/share/virtualenvs contains both Pipenv virtualenvs and virtualenvwrapper virtualenvs, so pyvenv-workon sees all of them.

I also noticed that I suddenly had a ~/.virtualenvs symlink pointing to ~/.local/share/virtualenvs, which something (maybe Elpy?) automatically created.

[jerryasher]

I'm a newbie to pipenv, but using emacs 25.3 on WSL (windows subsystem linux (ubuntu 16.04)), the latest pipenv (11.2.0) and the latest pipenv.el 0.0.1-beta https://github.com/pwalsh/pipenv.el, I find that m-x pipenv-activate once I have read a python file into emacs sets elpy correctly.

So for instance, m-x elpy-config reports in one of my repos...

Elpy Configuration

Virtualenv........: None
RPC Python........: 3.6.4 (/home/jerry/.local/share/virtualenvs/mcd-wK_c9w6V/bin/python)
Interactive Python: ipython (/home/jerry/.local/bin/ipython)
Emacs.............: 25.3.2
Elpy..............: 1.18.0
Jedi..............: 0.11.1
Rope..............: Not found (0.10.7 available)
Autopep8..........: 1.3.4
Yapf..............: 0.20.2
Syntax checker....: flake8 (/home/jerry/.local/bin/flake8)

You have not activated a virtual env. While Elpy supports this, it is
often a good idea to work inside a virtual env. You can use M-x
pyvenv-activate or M-x pyvenv-workon to activate a virtual env.

This is mostly correct. There is a virtualenv that elpy doesn't detect, however, with very limited testing, it seems the functionality from elpy, jedi, autopep8, yapf and flake8 seems to work.

I haven't tried @aparkerlue's suggestion of setting WORKON_HOME and then using pyvenv-workon.

One oddity that I am not sure how it should work, that is, what the recommended work flow is and what I should be doing instead, is that if I read a python file into emacs, then pipenv-activate the virtualenv, I need to revert buffer to get emacs to use flake8 analyze and annotate the file.

---

As an aside, I do wish there was someway I could tell elpy to use my virtualenv for the packages my specific python application requires, but to look at a different virtualenv to get the tools it needs for elpy itself, autopep8, yapf, flake8, elpy, jedi, (rope) from somewhere else.

When I just used pip I would install those as system site packages, and even with pipenv I can install them as site packages but in both cases it makes consequent uses of pip and pipenv a bit clunkier as I have to tell each for all other packages to ignore any other system site packages.

[jorgenschaefer]

Yeah, a nicer solution for that would be … nice. :-)

[aparkerlue]

A big part of doing things with Pipenv is being able to set secrets in a .env file. Could Elpy look for a .env file and add its contents to the environment before launching a Python shell?

This operation needs to be reversible so that exiting the shell and switching virtual environments doesn't retain variables from the old environment. Might be easier to make Elpy launch Python shells through Pipenv (i.e., pipenv run python/pipenv run ipython/pipenv run jupyter console). Perhaps:

(setq python-shell-interpreter "pipenv"
      python-shell-interpreter-args "run jupyter console --simple-prompt"
      python-shell-prompt-detect-failure-warning nil)
(add-to-list 'python-shell-completion-native-disabled-interpreters
             "pipenv")

Maybe this is all we need?

A couple .env-related projects for reference:

• kennethreitz/autoenv: Directory-based environments.
• preetpalS/emacs-dotenv-mode: An Emacs major mode for .env files

[memeplex]

I started using pipenv today, maybe it's because it's a new version of pipenv but I had not trouble making it work transparently with elpy. Calling pyenv-workon correctly shows my environments and selecting one of them just sets it without further hassle. Am I missing something?

[aparkerlue]

Here are two things that I think constitute Pipenv support in Elpy:

1. Provide a way to launch the desired Python interpreter through Pipenv, which loads .env environment variables.
2. Automatically identify and activate the associated virtualenv (pipenv --venv) when visiting a file.

(1) can be done per my last comment. Need (2).

[memeplex]

@aparkerlue what I'm saying is that I had to do literally nothing to get the following working:

1. pyvenv-workon lists the pipenv environments installed in my system.
2. I can select one of them.
3. Launching a python interpreter for a buffer (C-c C-z) starts it in the selected environment.

Does your first point imply more than that? Sorry, I had not used virtual envs with elpy before so I'm not sure what is missing.

[aparkerlue]

I think what you're saying is that you are able to get Elpy to recognize your Pipenv-installed virtual environment. That indeed works for me, too.

What Elpy does not do is

1. run the interpreter through Pipenv and
2. automatically activate the associated virtualenv when visiting a file.

For item (1), directly launching a Python/Jupyter/IPython console per the docs is fine, but launching the interpreter through Pipenv loads environment variables from .env (among whatever else Pipenv does, not my area of expertise). You can achieve a Pipenv launch per my June 21 comment, but my solution is a bit hacky: Elpy expects an interpreter, but Pipenv is not one. This is the resulting Elpy config output:

Elpy Configuration

Virtualenv........: project-abcdefgh (/Users/user/.virtualenvs/project-abcdefgh)
RPC Python........: 3.6.6 (/Users/user/.virtualenvs/project-abcdefgh/bin/python)
Interactive Python: pipenv (/Users/user/.local/bin/pipenv)
Emacs.............: 26.1
Elpy..............: 1.23.0
Jedi..............: 0.12.1
Rope..............: Not found (0.10.7 available)
Autopep8..........: 1.3.5
Yapf..............: 0.22.0
Black.............: 18.6b4
Syntax checker....: flake8 (/Users/user/.virtualenvs/project-abcdefgh/bin/flake8)

The python interactive interpreter (pipenv) is not installed on the
current virtualenv (/Users/user/.virtualenvs/project-abcdefgh/).
The system binary (/Users/user/.local/bin/pipenv) will be used instead.

[run] python -m pip install pipenv

Item (2) may or may not be within the scope of Elpy, though I think it would be awesome if it were.

[memeplex]

What Elpy does not do is run the interpreter through Pipenv and automatically activate the associated virtualenv when visiting a file.

Ok, but my question was more about whether these things are already supported for plain virtualenv or not. That is, is the issue addressed here pipenv specific or is it requesting to extend elpy to better integrate with virtual environments in general?

[jorgenschaefer]

run the interpreter through Pipenv

What, exactly, does running an interpreter through pipenv imply?

automatically activate the associated virtualenv when visiting a file.

Please note that virtualenvs are global in Emacs (due to the way environment variables work), not buffer-local, so this is tricky. pyvenv.el tries to do something like this for virtualenvwrapper projects, but it's quite annoying to use.

[aparkerlue]

What, exactly, does running an interpreter through pipenv imply?

That means calling pipenv run python, pipenv run ipython, or pipenv run jupyter console to invoke a Python interpreter using pipenv's do_run function (or so I believe), which loads the contents of the .env file into the environment before initiating the interpreter. Environment variables containing secrets and other configuration go in .env.

If automatically activating the right virtualenv isn't practically achievable, maybe it would suffice to provide a natural way to invoke the Python interpreter through Pipenv. There are active projects similar to Pipenv, such as Poetry and Hatch, so it may not make sense to go the whole nine yards with Pipenv just yet.

[jorgenschaefer]

Maybe the ability to set elpy-rpc-python-command to a list would work already? I'd be fine with such a change.

[rr326]

@jorgenschaefer Sorry if this is a dumb question. I've read through the thread and the docs. What is the recommended process for compling / testing / running a project that is set up with pipenv?

I was able to manually do an activate for the project and it works.

But is this how most people do it? I would really love some way to have it auto-activate for the project. Is this something I would set with projectile? Some sort of configuration? Really just to hard and I just have to manually set it up every time I want to compile for a certain project? It's hard for me to imagine emacs-ers would do this manually. I suspect there is a clever and easy way to do it, though as an emacs noob I'm lost.

Thanks in advance.

[galaunay]

Elpy does not specifically support pipenv, but I can see three ways to auto-activate pipenv virtualenv while using Elpy:

• Using pipenv.el and plugin it to projectile to switch virtualenv when switching project (see pipenv.el documentation here). The more robust option, but it will not work if you switch buffer with something else than projectile.
• Using pipenv.el and pyvenv-tracking-mode to automatically switch between virtualenvs. You will need something like this to plug pipenv.el to pyvenv:

(pyvenv-tracking-mode)
(defun pipenv-auto-activate ()
  "Set `pyvenv-activate' to the current pipenv virtualenv.

This function is intended to be used in parallel with
 `pyvenv-tracking-mode'."
  (pipenv-deactivate)
  (pipenv--force-wait (pipenv-venv))
  (when python-shell-virtualenv-root
    (setq-local pyvenv-activate
                (directory-file-name python-shell-virtualenv-root))
    (setq python-shell-virtualenv-root nil)))
(add-hook 'elpy-mode-hook 'pipenv-auto-activate)

It will work even without projectile, but as I just wrote the code and tested it quickly, it may still need some adjustments.

• Using pyvenv-tracking-mode and adding a .dirs-local.el file to all your projects. Those files will have to contain the path to the virtualenv to activate in a form looking like this:

((nil
  (pyvenv-workon . "/path/to/pipenv/virtualenv")))

The drawback being, you will need a .dirs-local.el file in all your projects...

I hope it helps, please let me know if you find another solution :).

[rr326]

Thanks @galaunay! I will try when I'm back in the project. Currently fighting other emacs showstopper challenges. ;)

[ltrojan]

no update on this? :(