nix-idea icon indicating copy to clipboard operation
nix-idea copied to clipboard

Published 20 hours ago • verified publisher icon NixOS

Add support for nix-shell in JetBrains products

Open jgeerds opened this issue 8 years ago • 27 comments

This might be a lot of work but it would be awesome if you could use nix-shell inside JetBrains products (e.g. activating shell.nix of your current project).

I have a lot of friends who use virtualenv instead of nix just because PyCharm (the python IDE by JetBrains) have builtin support for it.

What do you think about it?

jgeerds avatar Sep 14 '16 19:09 jgeerds

Yes of course, using profiles or shells for development is one of the main goals of this plugin.

edwtjo avatar Sep 18 '16 07:09 edwtjo

Hi,

Any update on this project?

afranche avatar Feb 25 '20 12:02 afranche

Maybe nix-shell --run 'env' > .env could be used with https://plugins.jetbrains.com/plugin/7861-envfile for the time being.

Mic92 avatar Feb 25 '20 12:02 Mic92

Just note that EnvFile is not currently supported across all JetBrains applications, unfortunately.

Most notably for me is CLion (for Rust envs, etc.)

Related: https://github.com/ashald/EnvFile/issues/71

evanjs avatar Feb 25 '20 13:02 evanjs

I generally start my pycharm while inside a nix-shell with my virtualenv. It works really well because pycharm is already able to reason about virtualenvs

jonringer avatar Feb 25 '20 16:02 jonringer

@jonringer - for the work-around win. :) Big thanks. I've been wracking my brain all afternoon trying to get it working. I'm now running NixOS on several machines and wanted to start using nix-shell instead of venv. Another vote for native support.

Some additional details to anyone still trying to figure out:

  1. Create you a default.nix/shell.nix in your Python project directory (if default.nix for packaging, you'll also probably want a setup.py in there also).
  2. Open terminal, nav to project dir, and type nix-shell to load up environment as normal
  3. Open PyCharm from within the nix-shell (ex: nohup pycharm-community >/dev/null 2>&1 & - so can close terminal after pycharm opens.)
  4. Add a Python Interpreter -> System Interpreter -> /run/current-system/sw/bin/python3.8 (assuming 3.8 on NixOS 20.09)
  5. Create you a Run Configuration pointing to the interpreter you just setup.
  6. Press the Run button and cross fingers. For me, main script ran as expected, debug seems to work, and PyCharm sees the packages pulled in from the shell.nix. That said though, I haven't tried to add more packages yet. We may have to update the shell.nix with any new packages and re-load PyCharm to add packages, that or maybe install same version in repo via PyCharm while programming and just remember to add it to shell.nix for next time.

ergose avatar Nov 27 '20 21:11 ergose

I created a plugin to load direnvs .envrc files at https://github.com/fehnomenal/intellij-direnv and use it with flakes and lorri in old projects.

It works, but is nothing but a big hack (because the JVM does not want you to modify the environment...). To natively support changing the environment on a per project basis, the IDE could clone the environment from the OS (like the JVM does it) and pass the clone to all processes it launches while allowing modifications. This would allow implementing direnv support without hacks.

fehnomenal avatar Jan 28 '21 12:01 fehnomenal

@ergose Thank you for the instructions! That wasn't quite enough to get IDEA to pick up the Python dependencies, I also had to add pip to the mkShell buildInputs, otherwise IDEA wouldn`t find the packages in the project SDK settings (and trying to install the packaging software within IDEA didn't work).

Unfortunately the /run/current-system/sw/bin/python… path does not exist when using Nix on Ubuntu.

Neither the envfile nor intellij-direnv seem to work for wiring up the SDK when using Nix outside of NixOS. The workaround is to change the path to the SDK every time the Nix shell is updated.

l0b0 avatar Sep 19 '21 23:09 l0b0

Hello,

I came across this plugin that tries to add this feature.
https://plugins.jetbrains.com/plugin/16864-enter-nix-shell You can find the source here: https://github.com/fctorial/enter_nix_shell

Maybe there are some interesting ideas that could be exploited. Also, the new flakes and nix develop environments could be a good option to add too.

SDAChess avatar Jan 31 '22 04:01 SDAChess

Thanks for your suggestions. I am reluctant to integrate the solution of Enter Nix Shell or intellij-direnv into this plugin. 😔 They both work by changing the environment of the IDEA process. This means that you cannot have different environments per project. While changing the IDEA environment with these plugins might be easier than restarting the whole IDE, it might also work less reliable. For example, see this page from Enter Nix Shell (archive.org). Besides that, you could also end up with a mix of multiple environments, which could have strange effects in some edge cases. While you might very well use one of these plugins, I feel like the implementations are too unstable for an official solution. At least that is my personal opinion.

JojOatXGME avatar Feb 06 '22 13:02 JojOatXGME

I used this weekend to explore the IntelliJ Platform API for alternatives. Unfortunately, I haven't found a good solution.

Run targets

I found Run targets (extension point com.intellij.executionTargetType). They allow users to execute run configurations within Docker containers or within WSL. It might also be usable to move the execution into a nix-shell. However, run targets are an ultimate feature and therefore not available for free. Also note that they do not affect the console panel or SDK detection. So it would only be a partial solution.

Run configuration extensions

EnvFile and ProjectEnv use com.intellij.​run​Configuration​Extension among other extension points. However, this extension point is specifically for IntelliJ (i.e. Java). Other IDEs have their own extension points. So we would have to implement it for every IDE separately. It is also not sure if all these extension points combined cover every IDE and type of run configuration.

Program runner

I also found the extension point com.intellij.programRunner. Program runners are the components which implement the execution of specific run configurations. I tried adding a custom program runner which claims it can execute all run configurations, which then modifies the environment and forwards it to the actual runner. Unfortunately, it looks like I cannot change environment variables at this level. I also couldn't make the runner work reliable in my tests. In many cases, IntelliJ continued to call the actual runner directly. Besides, this solution is also a bit hacky. I could imagine that it breaks some configurations.

Source code

plugin.xml

<extensions defaultExtensionNs="com.intellij">
  <programRunner implementation="org.nixos.idea.shell.NixProgramRunner" order="first"/>
</extensions>

NixProgramRunner.java

public final class NixProgramRunner implements ProgramRunner<RunnerSettings> {
    @Override
    public @NotNull @NonNls String getRunnerId() {
        return "nix-shell-wrapper";
    }

    @Override
    public boolean canRun(@NotNull String executorId, @NotNull RunProfile profile) {
        return true;
    }

    @Override
    public void execute(@NotNull ExecutionEnvironment environment) throws ExecutionException {
        ProgramRunner<?> runner = ProgramRunner.PROGRAM_RUNNER_EP.findFirstSafe(r -> matches(environment, r));
        // Somehow modify the environment?
        runner.execute(environment);
    }

    private boolean matches(@NotNull ExecutionEnvironment environment, @NotNull ProgramRunner<?> runner) {
        String executorId = environment.getExecutor().getId();
        return runner != this && runner.canRun(executorId, environment.getRunProfile());
    }
}

Execution target provider

The extension point com.intellij.executionTargetProvider also got my attention. However, on their own, execution targets only affect the UI. They seem to be used by Android Studio to choose the device where the app gets executed. But the implementation which looks at the configuration actually sits within the specific program runner (see last section) for Android apps. Adding an execution target causes them to be visible in the UI, but it does not impact the behavior at all.

grafik

Conclusion

As written in the first paragraph of this comment, it looks like there is no straightforward solution. From the options I have covered, the approach of EnvFile and ProjectEnv seems to be the most promising, but it might take a lot of effort. So I am still open for suggestions.

JojOatXGME avatar Feb 06 '22 19:02 JojOatXGME

I created a post on the community board of JetBrains. Let's see if someone responds. https://intellij-support.jetbrains.com/hc/en-us/community/posts/4418065675538-Project-specific-environment-variables Besides that, if someone knows the process to contribute new extension points to the IntelliJ Platform, I would be interested.

JojOatXGME avatar Feb 11 '22 19:02 JojOatXGME

Yeah, you did the same research as me. The main problem is that the JVMs have no way of modifying the environment (at least not with a public API).

Over a year ago I created a very similar suggestion to yours at https://youtrack.jetbrains.com/issue/IDEA-260725 but nothing happened, so I would not have high hopes... At least someone got assigned I guess :shrug:

For reference: https://youtrack.jetbrains.com/issue/IDEA-288756

fehnomenal avatar Feb 22 '22 05:02 fehnomenal

FYI Enter Nix Shell is not just abandoned, none of the links (source, issues, documentation) work anymore. Does anyone have a copy of the source code, so we could revive it?

l0b0 avatar Jun 01 '22 03:06 l0b0

FYI Enter Nix Shell is not just abandoned, none of the links (source, issues, documentation) work anymore. Does anyone have a copy of the source code, so we could revive it?

Interesting. It was still available in February. Unfortunately, I don't have a local copy of the repository. You can still download the binaries from JetBrains Marketplace (https://plugins.jetbrains.com/plugin/16864-enter-nix-shell) and use a decompiler. According to the page on the Marketplace, the plugin was available under MIT license.

I guess I should invalidate the links in my messages since they seem to point to some NSFW-site now.

JojOatXGME avatar Jun 01 '22 20:06 JojOatXGME

EnvFile and ProjectEnv use com.intellij.​run​Configuration​Extension among other extension points. However, this extension point is specifically for IntelliJ (i.e. Java). Other IDEs have their own extension points. So we would have to implement it for every IDE separately. It is also not sure if all these extension points combined cover every IDE and type of run configuration.

Hmm it seems like runConfigurationExtension is not available in IDEA 2022.2? Can't really find a way to extend new env in the new API

infiniteregrets avatar Aug 05 '22 20:08 infiniteregrets

Hmm it seems like runConfigurationExtension is not available in IDEA 2022.2? Can't really find a way to extend new env in the new API

Note that you have to add com.intellij.java as dependency because com.intellij.​run​Configuration​Extension is specific to IntelliJ and does not exist in other IDEs of JetBrains.

JojOatXGME avatar Aug 11 '22 21:08 JojOatXGME

FYI Enter Nix Shell is not just abandoned, none of the links (source, issues, documentation) work anymore. Does anyone have a copy of the source code, so we could revive it?

I've had a bit of an internet historian moment, and found the website of the developer of this plugin saved on web archive. It might have some interesting information and it also has some videos showing the plugin.

https://web.archive.org/web/20220131211828/https://fctorial.com/posts/nix_intellij.html

qlonik avatar Oct 07 '22 01:10 qlonik

@jonringer - for the work-around win. :) Big thanks. I've been wracking my brain all afternoon trying to get it working. I'm now running NixOS on several machines and wanted to start using nix-shell instead of venv. Another vote for native support.

Thank you for this! It does seem to break when changing the packages in the Nix shell, though, in a weird way:

Screenshot from 2023-01-10 13-49-51

Notice how, in the above, aws_cdk is marked as unknown even though aws-cdk-lib (which provides aws_cdk) is shown in the list of SDK packages. This inconsistency persists across IDEA restarts and even a full "Repair IDE" cycle. It's like IDEA no longer knows about the associated SDK. Even creating a new SDK pointing to /run/current-system/sw/bin/python doesn't seem to work.

Reported

l0b0 avatar Jan 10 '23 00:01 l0b0

I use Better Direnv and it seems to work:

  • https://github.com/nix-community/nix-direnv
  • https://plugins.jetbrains.com/plugin/19275-better-direnv

starquake avatar Apr 07 '23 12:04 starquake

@starquake One problem with that approach is that IDEA gets very confused about which packages are installed. This also happens when using nix-direnv + Better Direnv. We need a solution which works with more than one project.

l0b0 avatar Apr 11 '23 23:04 l0b0

Would it be possible to do something like

nix-shell --run ./ide_launcher.sh & disown
exit

to put the whole ide in the new environment?

Problems I see with this:

  • args to the launch script aren't preserved which could break things (eg remote dev)
    • possible fix; hook com.intellij.idea.Main or read args the normal java way (a quick skim through the launch script shows that no args go to the script itself)
  • If you want to switch between environments quickly, you'll have to wait for the IDE to restart every time
  • There might not be a way to easily make the whole IDE exit

GenericNerdyUsername avatar Jul 17 '23 14:07 GenericNerdyUsername

Here's a summary of the most significant things mentioned:

nhooey avatar Aug 20 '23 18:08 nhooey

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/jetbrains-ides-support-for-nix-shell-w-nix-idea-plugin-per-project-environment/32321/1

nixos-discourse avatar Aug 29 '23 15:08 nixos-discourse

hey we patch the env of all the intellij IDEs, maybe this approach could help?

https://github.com/metalbear-co/mirrord-intellij

infiniteregrets avatar Aug 29 '23 16:08 infiniteregrets

Here's a workaround, after trying many things. tl;dr: Symlink the python executable somewhere static, then point the SDK to that path.

Tested only with IDEA and poetry2nix repos.

Caveats:

  • IDEA seems to require that you include pip in the Python derivation, or else it doesn't find the installed packages.
  • If you're using direnv you have to symlink from both .envrc and shell.nix.
  • IDEA does not seem to pick up on environment changes immediately, and requires a restart for new packages to show up.

Example repos: 1, 2.

l0b0 avatar Dec 19 '23 01:12 l0b0

@infiniteregrets: hey we patch the env of all the intellij IDEs, maybe this approach could help?

https://github.com/metalbear-co/mirrord-intellij

@infiniteregrets: Could you elaborate on your approach, or even highlight where in the mirrord JetBrains extension code the environment patching is implemented?

nhooey avatar May 21 '24 07:05 nhooey