SOLR-14414: Introduce new UI (SIP-7)

[malliaridis]

https://issues.apache.org/jira/browse/SOLR-14414 Latest discussion thread: https://lists.apache.org/thread/knd5xgkswp83x92c0sclh7ghcob7bpfx Proposal: SIP-7

Description

Due to the end-of-life of AngularJS (2022) we have to introduce a new Admin UI implementation that fully replaces the current webapp. SIP-7 as well as SOLR-14414 address this matter.

Solution

The proposed solution in this PR is an implementation of a new UI based on Kotlin and Compose Multiplatform that allows an integration of the new UI in the current webapp (until sufficient features are available to remove the old UI) as a WebAssembly app. The application can also be executed as a standalone desktop client (JVM-based) via gradlew :solr:compose-ui.

Important This is a draft PR to give maintainers the option to review the changes. The PR is not ready for merge yet.

Noticable (new) features:

• Theme support - currently the dark/light theme is automatically selected based on device configration
• Internationalization / Localization - Translations can be provided as simple XML files
• Unified patterns - Each UI component follows the same implementation patterns to allow an easier start for new developers
• Improved UI - New Designs have been created to solve various UX issues reported in the past
• Version Catalog - Currently only used by new UI

For a better understanding of the files and classes see added dev-docs.

Tests

Tests will be provided soon.

Checklist

• [X] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
• [ ] I have created a Jira issue and added the issue ID to my pull request title.
• [X] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
• [X] I have developed this patch against the main branch.
• [ ] I have run ./gradlew check.
• [ ] I have added tests for my changes.
• [ ] I have added documentation for the Reference Guide

[gerlowskija]

Also - when I check out this code locally it compiles fine in my IDE, but I'm having trouble running the packaging tasks (e.g. ./gradlew assemble) in my terminal. The gradle build hangs on several of the compose-ui tasks. In the example below I let it run > 50m before killing the build:

> Task :solr:compose-ui:compileKotlinWasmJs
Options for KOTLIN DAEMON: IncrementalCompilationOptions(super=CompilationOptions(compilerMode=INCREMENTAL_COMPILER, targetPlatform=JS, reportCategories=[0, 3], reportSeverity=2, requestedCompilationResults=[0], kotlinScriptExtensions=[]), areFileChangesKnown=false, modifiedFiles=null, deletedFiles=null, classpathChanges=NotAvailableForJSCompiler, workingDir=/Users/gerlowskija/checkouts/solr/solr/compose-ui/build/kotlin/compileKotlinWasmJs/cacheable, multiModuleICSettings=MultiModuleICSettings(buildHistoryFile=/Users/gerlowskija/checkouts/solr/solr/compose-ui/build/kotlin/compileKotlinWasmJs/local-state/build-history.bin, useModuleDetection=false), usePreciseJavaTracking=false, icFeatures=IncrementalCompilationFeatures(withAbiSnapshot=false, preciseCompilationResultsBackup=true, keepIncrementalCompilationCachesInMemory=true, enableUnsafeIncrementalCompilationForMultiplatform=false), outputFiles=[/Users/gerlowskija/checkouts/solr/solr/compose-ui/build/classes/kotlin/wasmJs/main, /Users/gerlowskija/checkouts/solr/solr/compose-ui/build/kotlin/compileKotlinWasmJs/cacheable, /Users/gerlowskija/checkouts/solr/solr/compose-ui/build/kotlin/compileKotlinWasmJs/local-state])
<===========--> 90% EXECUTING [51m 4s]
> IDLE
> IDLE
> :solr:compose-ui:compileCommonMainKotlinMetadata
> :solr:compose-ui:compileKotlinDesktop
> :solr:compose-ui:compileKotlinWasmJs
> IDLE

Going to try this on different machines and hopefully work around it. But if anyone has any insights (or even specific commands that worked for them), I'd love to get this working locally to kick the tires of the new UI.

[malliaridis]

2. The biggest downside in the code for me is how boilerplate heavy it is. There are Component interfaces, which have Component implementations, which then reference Models, which come from Stores, which fetch the data via API calls. Those abstractions all make sense individually. But they're the main reason probably that this turned into an 8k line PR!

You are right. There are multiple layers that abstract API, logic and UI from each other, resulting in a lot of boilerplate code. The repetitiveness is also a side-effect of applying the same patterns to all componets.

Besides the drawback of the quantity in changes, there are two main benefits that may be achieved with that approach: testing components / individual parts becomes much easier, and new developers have an easier start because they can follow the structure and patterns from other components when they are working on a new component.

I wanted to write some tests for completion, but I had to postpone it for the time being.

Positing community consensus on taking this forward, I think we'll need to break this into smaller pieces that are less intimidating to folks. I suspect you've already thought a bit about where to draw some of those lines, but I have at least some hunches we could talk over at some point. (Split "Java Properties" and "logging" pages into different PRs, slim the theme down to only support a single mode, commit the theme separately, etc.)

You are completely right on that. It is possible, and this was the plan once we work on the Jira part of the migration, to split the components into separate PRs and tickets. This would allow in the long run an easier integration of new components. Contributors will also have to worry only about a single component and not the entire application when working on a ticket, PR, feature or bugfix.

The POC includes already two of the components for demonstration, so that it covers almost all parts of the Compose app (including navigation) that are crucial to get started. The only thing I can recall at the moment and that is missing is probably the explicit error handling, but that is something I have already worked on in other projects and shouldn't be a problem integrating.

I've left a few comments inline.

Thank you very much for your time reviewing this. As you noticed, this POC still needs some work and cleanup before moving forward.

[epugh]

I am getting an error when running ./gradlew dev:

Execution failed for task ':solr:compose-ui:compileProductionExecutableKotlinWasmJs'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Not enough memory to run compilation. Try to increase it via 'gradle.properties':
     kotlin.daemon.jvmargs=-Xmx<size>

Going to try and change that property!

[malliaridis]

@gerlowskija and @epugh the build errors you have probably occur because of the jvmargs in gradpe.properties that need to be updated. I added -Dkotlin.daemon.jvm.options\="-Xmx2048M" to org.gradle.jvmargs. If I remember correct, I had similar issues without it.

org.gradle.jvmargs=-Xmx2g -XX:TieredStopAtLevel=1 -Dfile.encoding=UTF-8 -Dkotlin.daemon.jvm.options\="-Xmx2048M" -XX:+UseParallelGC -XX:ActiveProcessorCount=1 \
 --add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
 --add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \
 --add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \
 --add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
 --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED

Additionally, I added kotlin.code.style=official for the code styling in the gradle.properties.

Starting with a clean build may also fix some issues, if it still doesn't build.

[epugh]

Changing to org.gradle.jvmargs=-Xmx2g in gradle.properties allowed the dev target to complete!

[epugh]

I got it to run! http://localhost:8983/solr/compose! So, it might be a bit of bike shedding, but I wonder if it should just be called ui? Just to distingush it from admin? Or maybe just refer to it as compose everywhere, instead of compose-ui?

So, thinks I would like to see are:

1. How does this handle authentication/authorization. I'm hoping we have some really nice support that can leverage what solr already has.
2. I'm very curious to see "how easy is it to expose a simple V2 Rest api". I'm hopeful that we can leverage the great v2 openapi work to get a lovely kotlin client (instead of hand writing it like you had to for the Java properties). --> This would help me better understand how we can get more admin tool with less work!
3. if a core aspect is running it as a local client, that connects to a remote Solr (as well as a web app!), then I'd love to see more about how that app might be packaged up and distributed and connect to arbitrary Solr!
4. Lastly, if there is a lot of pushback on shipping another large webapp in solr, could this app be distributed as a Solr package? Similar to how https://github.com/o19s/splainer?tab=readme-ov-file#splainer-package-for-solr packages up a angularjs app for Solr.

Very encouraging!

[malliaridis]

How does this handle authentication/authorization. I'm hoping we have some really nice support that can leverage what solr already has.

I am eager to imlpement it since the first day you requested this, and will may start working on it soon. But I will probably not add it to the current POC because it will be too much to review.

I'm very curious to see "how easy is it to expose a simple V2 Rest api". I'm hopeful that we can leverage the great v2 openapi work to get a lovely kotlin client (instead of hand writing it like you had to for the Java properties). --> This would help me better understand how we can get more admin tool with less work!

A Kotlin client would be great to use here (it could eventually replace the current SolrJ implementation too). I have discussed some thoughts with @gerlowskija, but nothing noteworthy yet. I think if this POC gets accepted, we would have more reasons for introducing a additional admin client for Kotlin and support a bunch of new targets.

if a core aspect is running it as a local client, that connects to a remote Solr (as well as a web app!), then I'd love to see more about how that app might be packaged up and distributed and connect to arbitrary Solr!

I think you can already package / build an installer for your platform by using one of the gradle tasks provided in the group compose desktop, starting with packageRelease*. I haven't tested it yet for this build, so there may be some configurations necessary before a successful build.

Lastly, if there is a lot of pushback on shipping another large webapp in solr, could this app be distributed as a Solr package? Similar to how https://github.com/o19s/splainer?tab=readme-ov-file#splainer-package-for-solr packages up a angularjs app for Solr.

This is something that still raises concerns. Replacing the current webapp is one of the main goals of the new UI, and if it is shipped separately / outside of Solr project, we may not be able to deprecate and replace the current webapp with it and we will end up with "yet another Solr Admin tool". I think that if we notice further down the path that it gets out of hand, we may reconsider it and eventually remove the new module / UI completely (however, this may be a topic for the mailing list).

I am not sure if that is what you meant, or if you simply want to see the UI as a plugin that can be loaded like any other plugin we have (sql, ltr etc).

[janhoy]

UI should absolutely be a standard component as always, imo not suited as a pkg. if we decide that solr should ship without a UI then that should be a separate decision, decoupled from this technology refresh. And if/when solr ships without a UI, then I believe the admin could be a separate app not running in the Solr JVM at all.

[gerlowskija]

Curious where we stand on this POC...

If I remember right, one of the main questions that needs answered at this point is: "Do we think UI code in Kotlin is a better fit for our community of backend developers than JS or another 'traditional' frontend language?"

I think I'm personally reassured on that front - even with this PR being massive I found it easier to understand than most JS I've spent time with. But it's a pretty big decision to make without more voices chiming in. @epugh @janhoy or others - do you guys have any thoughts/opinions on the language choice aspect of this?

[malliaridis]

I plan to prepare a presentation + meeting where I go through the code, explain a few things and eventually do some live coding to show how to read the code, structure things and get started with a new component, so that people get more comfortable and give it a try themselves. I am just a bit short in time these weeks to prepare and announce something.

Hopefully this helps in the decision.

[epugh]

I've poked around a bit and am very interested in seeing it move forward!

[epugh]

Checking code out... I got a

task :solr:compose-ui:compileProductionExecutableKotlinWasmJs FAILED
e: java.lang.OutOfMemoryError: GC overhead limit exceeded
        at org.jetbrains.kotlin.wasm.ir.convertors.WasmIrToBinary.appendInstr(WasmIrToBinary.kt:256)
        at org.jetbrains.kotlin.wasm.ir.convertors.WasmIrToBinary.appendExpr(WasmIrToBinary.kt:477)

with the suggestion

 Not enough memory to run compilation. Try to increase it via 'gradle.properties':
     kotlin.daemon.jvmargs=-Xmx<size>

bumping to 2G worked.

[epugh]

i expected the link to the /compose to work from old admin, but didn't see it...

[malliaridis]

i expected the link to the /compose to work from old admin, but didn't see it...

The old UI only links to the new UI if you go to "Java Properties". The idea is to have a button only in places where a migration in the new UI is available. The URL the new UI is running is right now http://localhost:8983/solr/compose.

Alternatively you can always run gradlew :solr:compose-ui:run to launch the desktop client...

[epugh]

I think having a single link from the top of the old admin to the new ui is fine. I am hoping that as part of the new UI, you don't feel constrained to copy the old Admin UI. It's okay to rethink things, like maybe putting Collections first, and sub oridinating Cores below them.

I also think a way to get buyin and more excitement is to demonstrate by adding new Admin features compared to what exists. For example, I find using the https://solr.apache.org/guide/solr/latest/configuration-guide/config-api.html tedious and error prone. It would be amazing if you were able to add that support in the UI. Get new value sooner. Which also might help get new eyes on things sooner.