Optimize project for development with IDEs

[philsttr]

Is your feature request related to a problem? Please describe.

IDEs really struggle with this project since it is so huge and has so many submodules with many dependencies.

I believe this is a known problem (https://github.com/open-telemetry/opentelemetry-java-instrumentation/discussions/9331, https://github.com/open-telemetry/opentelemetry-java-instrumentation/discussions/8767), but I am filing this issue to bring more attention to it, and maybe track some progress.

Please take this as constructive feedback, and not criticism, because I'm really excited for this project, and I fully understand how it has grown to become this behemoth.

This sounds harsh, but... development on this project is so painful that I am questioning whether I want to contribute more to this project (I'm considering contributions for https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/9740, https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/9400, and https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/9390), or if I should maintain custom in-house extensions instead. I would much prefer to contribute to this project, but after comparing my experience contributing to it (https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/9705) versus my experience developing some separate in-house extensions for proprietary code... I can say that my experience developing in-house extensions was much easier/faster, mainly due to how much IDEs struggle with this project. IDEs constantly index and run out of memory (even giving them 8+GB), making iterative development terribly slow. I've resorted to editing files in a basic text editor, which is not a great experience after working with IDEs for years.

Describe the solution you'd like

The ability to perform iterative development on this project using an IDE with a reasonable footprint. For example, lower memory and CPU requirements, less indexing, faster builds, etc.

Describe alternatives you've considered

Don't use an IDE. Use a basic text editor instead.

Don't contribute to this project. Instead, develop custom extensions in separate repos.

Additional context

Reduce developer power bills by not making our computers work so hard. ;) On the bright side, since I have to let my computer cool off frequently when developing on this project, it forces me to take more breaks.

[trask]

hi @philsttr, thanks for opening this and bringing attention back to it.

we discussed in our weekly meeting, and have a couple of ideas about trying to split up the project into multiple gradle modules to allow loading only the piece that you are working on into Intellij

we probably won't be able to get to this until after we release version 2.0 (which is currently targeted for Dec or Jan) for a couple reasons:

• With 2.0, HTTP semantic conventions will be stable, which will cover a lot of our instrumentations, and should reduce our need to do large scale refactoring across tons of modules (in case we aren't able to find a way to be able to support loading modules individually and collectively)
• We are currently consumed time/effort-wise with getting 2.0 out, and so the maintainers probably won't have time to put into this before then (though if anyone else is able to play around and try stuff out earlier that would be great)

@zeitlinger also opened https://intellij-support.jetbrains.com/hc/en-us/community/posts/14704481872914-Huge-Java-project-in-IDEA in case we can get any other ideas / tips on how to make the situation more bearable

[laurit]

If you are interested in working only one a subset of instrumentations you could try commenting out most of the instrumentations in settings.gradle.kts. It is a bit of a hit and miss because some stuff depends on other stuff but not too hard. Try the following, comment out include(":smoke-tests-otel-starter") this depends on spring instrumentations. Comment out all other instrumentations except the ones starting with :instrumentation:internal, :instrumentation:executors, :instrumentation:external-annotations and :instrumentation:opentelemetry-. Now try building the project and see if works. If it does add the instrumentations you wish to work on along with their dependencies.

[tylerbenson]

Another alternative to the above suggestion is to Unload Modules for instrumentation you don't need/care about. https://www.jetbrains.com/help/idea/unloading-modules.html

[philsttr]

Oh nice. I was not aware of the Unload Module feature. That sounds like a decent solution for IntelliJ. I'll give it a shot the next time I'm working on this project.

[philsttr]

Unloading Modules in IntelliJ has been a huge help. I've submitted https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/9810 to add this info to the IntelliJ Setup and Troubleshooting doc.

Thanks for pointing it out @tylerbenson

[tylerbenson]

Thanks for the feedback and PR @philsttr!

[jaydeluca]

does anyone think this issue is still needed at this point? I haven't had any problems but I'm also on a pretty powerful machine

[github-actions[bot]]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed automatically if there is no response from the author within 7 additional days from this comment.