please support use of the system black installation

Is your feature request related to a problem? Please describe.

As a new user of black, I installed the black package from my distribution to try it out. After that, I wanted to integrate it into vim. I set up the plugin, but discovered that the plugin requires building a virtualenv within which to install black. Using a virtualenv is undesirable to me for the following reasons:

• I already have black installed; there should be no need to install it again.
• The virtualenv adds clutter to my ~/.vim directory.
• I'd prefer to upgrade/manage black via my distribution's package manager.
• In order to use a virtualenv, I would need to install additional python packages for which I have no other immediate use.

Describe the solution you'd like

I'd prefer the vim plugin to use the system installation of black. Since having a virtualenv is apparently desirable to other people, this should be configurable.

Describe alternatives you've considered

I've considered using the virtualenv.

Additional context

Please review the PR which I will submit shortly.

Thanks, Corey

Doesn't setting g:black_virtualenv = "/usr/local/" somewhere in vim startup files (or specifically for python files) do that? The path may vary depending on your distribution, it will search for the site-packages below that point (see _get_virtualenv_site_packages function)

Doesn't setting g:black_virtualenv = "/usr/local/" somewhere in vim startup files (or specifically for python files) do that? The path may vary depending on your distribution, it will search for the site-packages below that point (see _get_virtualenv_site_packages function)

Hmm. I hadn't looked at that before now, but I don't think it's suitable. I'll elaborate below.

That function is:

def _get_virtualenv_site_packages(venv_path, pyver):
  if sys.platform[:3] == "win":
    return venv_path / 'Lib' / 'site-packages'
  return venv_path / 'lib' / f'python{pyver[0]}.{pyver[1]}' / 'site-packages'

...and it's called as (abridged):

  virtualenv_path = Path(vim.eval("g:black_virtualenv")).expanduser()
  virtualenv_site_packages = str(_get_virtualenv_site_packages(virtualenv_path, pyver))
  if not virtualenv_path.is_dir():
    # (set up black virtualenv)...

So it looks like g:black_virtualenv needs to refer to a directory that has subdirectories lib/python3.10/site-packages, at least for my python version.

This doesn't match anything in the Debian package structure.

$ dpkg -L black
/.
/usr
/usr/bin
/usr/bin/black
/usr/lib
/usr/lib/python3
/usr/lib/python3/dist-packages
/usr/lib/python3/dist-packages/_black_version.py
/usr/lib/python3/dist-packages/black
/usr/lib/python3/dist-packages/black/__init__.py
/usr/lib/python3/dist-packages/black/__main__.py
/usr/lib/python3/dist-packages/black/brackets.py
/usr/lib/python3/dist-packages/black/cache.py
/usr/lib/python3/dist-packages/black/comments.py
/usr/lib/python3/dist-packages/black/concurrency.py
/usr/lib/python3/dist-packages/black/const.py
/usr/lib/python3/dist-packages/black/debug.py
/usr/lib/python3/dist-packages/black/files.py
/usr/lib/python3/dist-packages/black/handle_ipynb_magics.py
/usr/lib/python3/dist-packages/black/linegen.py
/usr/lib/python3/dist-packages/black/lines.py
/usr/lib/python3/dist-packages/black/mode.py
/usr/lib/python3/dist-packages/black/nodes.py
/usr/lib/python3/dist-packages/black/numerics.py
/usr/lib/python3/dist-packages/black/output.py
/usr/lib/python3/dist-packages/black/parsing.py
/usr/lib/python3/dist-packages/black/py.typed
/usr/lib/python3/dist-packages/black/report.py
/usr/lib/python3/dist-packages/black/rusty.py
/usr/lib/python3/dist-packages/black/strings.py
/usr/lib/python3/dist-packages/black/trans.py
/usr/lib/python3/dist-packages/black-22.8.0.dist-info
/usr/lib/python3/dist-packages/black-22.8.0.dist-info/AUTHORS.md
/usr/lib/python3/dist-packages/black-22.8.0.dist-info/METADATA
/usr/lib/python3/dist-packages/black-22.8.0.dist-info/RECORD
/usr/lib/python3/dist-packages/black-22.8.0.dist-info/WHEEL
/usr/lib/python3/dist-packages/black-22.8.0.dist-info/entry_points.txt
/usr/lib/python3/dist-packages/black-22.8.0.dist-info/top_level.txt
/usr/lib/python3/dist-packages/blackd
/usr/lib/python3/dist-packages/blackd/__init__.py
/usr/lib/python3/dist-packages/blackd/__main__.py
/usr/lib/python3/dist-packages/blackd/middlewares.py
/usr/lib/python3/dist-packages/blib2to3
/usr/lib/python3/dist-packages/blib2to3/Grammar.txt
/usr/lib/python3/dist-packages/blib2to3/PatternGrammar.txt
/usr/lib/python3/dist-packages/blib2to3/__init__.py
/usr/lib/python3/dist-packages/blib2to3/pgen2
/usr/lib/python3/dist-packages/blib2to3/pgen2/__init__.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/conv.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/driver.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/grammar.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/literals.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/parse.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/pgen.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/token.py
/usr/lib/python3/dist-packages/blib2to3/pgen2/tokenize.py
/usr/lib/python3/dist-packages/blib2to3/pygram.py
/usr/lib/python3/dist-packages/blib2to3/pytree.py
/usr/share
/usr/share/bash-completion
/usr/share/bash-completion/completions
/usr/share/bash-completion/completions/black
/usr/share/doc
/usr/share/doc/black
/usr/share/doc/black/README.Debian
/usr/share/doc/black/changelog.Debian.gz
/usr/share/doc/black/changelog.gz
/usr/share/doc/black/copyright
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/black-primer.1.gz
/usr/share/man/man1/black.1.gz

I can trick the plugin by making an empty dir, e.g.:

mkdir -p /tmp/black/lib/python3.10/site-packages/

...and then setting in vim:

:let g:black_virtualenv = "/tmp/black"

...but that seems hackish, and it will break upon my next python version upgrade or if the plugin is ever modified to do any other validation of the virtualenv.

g:black_virtualenv = "/usr/local/" worked (at least for me) under 22.8.0, but failed like you predicted with 22.10.0 requiring access python site-packages directory.

It's (at least partly) related to the CACHE_DIR (or setting BLACK_CACHE_DIR environment variable). ... File "/usr/local/lib/python3.9/site-packages/black/nodes.py", line 25, in <module> pygram.initialize(CACHE_DIR) ... FileNotFoundError: [Errno 2] No such file or directory: /usr/local/lib/python3.9/sitepackages/blib2to3/home/rkruus/.cache/black/22.10.0/

That cache is now being being forced under the site-packages directory, even when setting BLACK_CACHE_DIR -- BLACK_CACHE_DIR is appended to site-packages. Explicitly setting that variable that uses an absolute path may work? Or test for write in the CACHE_DIR and fallback to $HOME/.cache/black?

The previous default behaviour was to to use (as can be seen in the error message above): $HOME/.cache/black/VERSION

g:black_virtualenv = "/usr/local/" worked (at least for me) under 22.8.0, but failed like you predicted with 22.10.0 requiring access python site-packages directory.

Not to contradict you, but to clarify a bit, the failure I'm getting happens earlier than what you are seeing.

For your environment, this line returns a directory that exists:

return venv_path / 'lib' / f'python{pyver[0]}.{pyver[1]}' / 'site-packages'

For my environment, no such directory exists.

Either way, I think that g:black_virtualenv is too fragile to be a usable workaround.