Installing plugin as dist or local path into Poetry's environment

[jo014670]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10
• Poetry version: 1.2.0
• Link of a Gist with the contents of your pyproject.toml file: N/A

Issue

There does not appear to be a way to install a custom plugin locally without involving PyPi. I've tried the following commands:

Within the plugin directory:

poetry self add <plugin name>

Result:

'NoneType' object has no attribute 'startswith'

Within the dist directory after building:

poetry self add <wheel file>

Result:

Could not parse version constraint: <wheel file>

I have been able to work around this by using the system pip to install the plugin, but this is an ugly workaround for an issue that makes testing incredibly difficult.

[neersighted]

Currently poetry self add is designed solely for convenience/the end user. For development, I would suggest using the copy of pip bundled in Poetry's environment.

e.g.

~/.local/share/pypoetry/venv/bin/pip install <path to wheel>

This could easily become a docs ask -- I don't think we'll change how poetry self add requires a constraint as it is fundamentally based on regular poetry add and manages an internal pyproject.toml (so that one can configure plugins as part of their dotfiles and have Poetry install them in one command).

[jo014670]

This is what I do:

<user directory>/AppData/Roaming/pypoetry/venv/Scripts/pip.exe install <path to wheel>

Which succeeds, but:

poetry show <plugin>

Result:

Package <plugin> not found

[neersighted]

You should be using poetry self show to introspect Poetry's internal environment

[jo014670]

Yes, good point. I do that, and:

Error: poetry.lock not found. Run `poetry lock` to create it.

So I run poetry self lock with success. However:

> poetry self show <plugin name>
Package <plugin name> not found

Also worth noting is using the workaround approach, the custom commands show up in poetry list, however using Poetry's internal environment, it does not.

[neersighted]

Ah, show is based on the lock file and not solely iterating over the environment -- it will indeed not be great for this use case. Good old pip list is probably the best way to reason about this right now.

[neersighted]

Also worth noting is using the workaround approach, the custom commands show up in poetry list, however using Poetry's internal environment, it does not.

Not sure what this means -- could you clarify?

[jo014670]

Also worth noting is using the workaround approach, the custom commands show up in poetry list, however using Poetry's internal environment, it does not.

Not sure what this means -- could you clarify?

I am able to get my custom plugin to work using the system pip to install it, but this is an ugly workaround approach. When I do that, the custom commands associated with my plugin show up in poetry list. However, using Poetry's pip to do so, it does not.

[jo014670]

Ah, show is based on the lock file and not solely iterating over the environment -- it will indeed not be great for this use case. Good old pip list is probably the best way to reason about this right now.

But I created a lock file for Poetry's internal environment using poetry self lock...so how is poetry self show not appropriate for this?

[neersighted]

Because the lock file is based on the contents of the internal pyproject.toml and not anything installed into the environment using external tooling -- which the internal pip is. The primary intention of poetry self add is for end-users to install a package from a index and not so much for development usage, but it shouldn't be too hard to generalize it more to match the regular poetry add command and support more dependency types to improve ergonomics for development.

[jo014670]

So, I guess that goes back to my original problem, how is someone expected to create and test plugins before publishing them?

[neersighted]

Using pip is good enough for development even if not ergonomic, and then poetry self can be used by end users. Improving poetry self for development would be nice, but was not part of the MVP of plugins and would be part of a future release (e.g. 1.3 if a PR materialized).

[jo014670]

That makes sense, but plugins do not appear to work when installed with Poetry's pip. And as it is right now poetry self cannot install plugins from non-PyPi sources, so it seems it cannot be used by all end users either.

[neersighted]

Poetry's pip should work -- are you sure you defined the entrypoint on your plugin? Poetry discovers plugins by iterating over entrypoints and importing tool.poetry.plugins entrypoints.

[jo014670]

Are you referring to this?

[tool.poetry.plugins."poetry.plugin"]
demo = "poetry_demo_plugin.plugin:MyPlugin"

Is that just supposed to be defined in the plugin repo? I install it from within that repo (using Poetry's pip) and it succeeds, and I can see it on Poetry's pip show, but Poetry still cannot find my custom commands, even within the repo with that entrypoint defined.

[neersighted]

It is supposed to be on the plugin pyproject.toml only -- I am unable to reproduce your issue with plugins not being detected by Poetry.

e.g.

$ ~/.local/share/pypoetry/venv/bin/pip install poetry-plugin-bundle
[...]
$ ~/.local/share/pypoetry/venv/bin/pip list
[...]
poetry-plugin-bundle 1.0.0
[...]
$ ~/.local/share/pypoetry/venv/bin/poetry list
[...]
 bundle
  bundle venv        Bundle the current project into a virtual environment
[...]
$ ~/.local/share/pypoetry/venv/bin/poetry self show plugins
[...]
  • poetry-plugin-bundle (1.0.0) Poetry plugin to bundle projects into various formats
      1 application plugin

      Dependencies
        - poetry (>=1.2.0rc1,<2.0.0)
[...]
$ ~/.local/share/pypoetry/venv/bin/poetry bundle venv target
[success]

[neersighted]

Looking closer, you need to use an application plugin entrypoint/the correct class, per the docs:

[tool.poetry.plugins."poetry.application.plugin"]
foo-command = "poetry_demo_plugin.plugin:MyApplicationPlugin"

[jo014670]

I see, looks like I had two poetry installations that were competing with each other, one installed with the script and one installed via the system pip. The poetry pip I was using to install the plugin was not the same poetry as the one I was using to do poetry list, which means it was not found. Thanks for your help and quick response!

One thought I did have, you said poetry self add is meant for the end user. However, as it is right now, you can't use any other source than PyPi, which limits the end users that can use that command. Is there any thought as to adding support for using that command with other repositories in the future, like with a poetry self source add or something like that?

[neersighted]

There's certainly room to expand it, but I don't think we want to do so before we rework source handling as discussed in some PRs/issues on this tracker. In the meantime, as most plugins will install fast and easily enough as a sdist, teaching poetry self add to accept more inputs (like poetry add) might be a better solution.

[neersighted]

Reopening to track the issue title

[nicoloboschi]

For reference, my quick solution to test a local branch of a plugin is:

$(poetry run which pip) install dist/<wheel>

[peterchristofferholm]

For the record, this seems to be working nicely.

pipx inject poetry path/to/plugin/ --editable