Add solr based autocomplete option to submission form input

References

Add references/links to any related issues or PRs. These may include:

• Fixes #9612

Description

This development implements an autocomplete feature on the new item submission form, utilizing existing metadata values from previously recorded items. Currently, the DSpace submission workflow supports autocomplete only with pre-defined values in an XML file. The solution involves making the desired behavior (solr-based autocomplete) configurable.

Instructions for Reviewers

List of changes in this PR:

• new ChoiceAuthority implementation (LocalMetadataLookup.java) to get solr-based autocompete values
• configurations in dspace/config/modules/authority.cfg to use this new feature. There are two new configuration key
  • choices.order: specifies the order in which the autocomplete results should appear in the dropdown. It can have two values: count (quantity-based order) or index (alphabetical order)
  • choices.index: specifies which field within the solr search core should the application use for autocomplete values
• theese two new configuration fields were added as constants to the ChoiceAuthorityServiceImpl.java among the other configuration key constants.

To test this feature, the commented configuration in the authority.cfg can be used. In this case, the autocomplete option will be available for the Author and Subject fields. It is also important that no vocabulary is defined for these inputs in the dspace/config/submission-forms.xml configuratuin file.

Checklist

This checklist provides a reminder of what we are going to look for when reviewing your PR. You need not complete this checklist prior to creating your PR (draft PRs are always welcome). However, reviewers may request that you complete any actions in this list if you have not done so. If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!

• [ ] My PR is created against the main branch of code (unless it is a backport or is fixing an issue specific to an older branch).
• [ ] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & integration tests). Exceptions may be made if previously agreed upon.
• [ ] My PR passes Checkstyle validation based on the Code Style Guide.
• [ ] My PR includes Javadoc for all new (or modified) public methods and classes. It also includes Javadoc for large or complex private methods.
• [ ] My PR passes all tests and includes new/updated Unit or Integration Tests based on the Code Testing Guide.
• [ ] My PR includes details on how to test it. I've provided clear instructions to reviewers on how to successfully test this fix or feature.
• [ ] If my PR includes new libraries/dependencies (in any pom.xml), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
• [ ] If my PR modifies REST API endpoints, I've opened a separate REST Contract PR related to this change.
• [ ] If my PR includes new configurations, I've provided basic technical documentation in the PR itself.
• [ ] If my PR fixes an issue ticket, I've linked them together.

@tdonohue I'd like to review this PR.

@damian-joz : Please feel free to review. You are welcome to review any PR(s) that are of interest to you / your team.

@tdonohue I'd like to review this PR.

I really thank you! If you find the solution good, is there a chance if this addition could be part of the DSpace 8.1 release?

Dear @damian-joz, Could you test this feature? Thank you!

I have uploaded a short video how it's working for example on dc.type or dc.subject fields. If you use this feature you can select values stored inside your DSpace database.

https://github.com/user-attachments/assets/5ed980d3-db38-4485-8ee0-8d234f3899e9

@kanasznagyzoltan Hi, I've tried to do a review but I can't make it running, config lines were uncommented at authority.cfg and project was rebuilded. Are there any other steps to make it work?

I just tested this pull request, and I have a few discoveries for this feature to make it work. I tried to implement this with a local metadata and I found out that I need to add search filters in the [dspace-install]/config/spring/api/discovery.xml. I'm not sure if this is relevant but I made sure that the value of indexFieldName should be equal to the choices.index before the _prefix.

For example, I have a local metadata field named local.subject.personalName, in my authority.cfg file, I have this entry:

## Metadata lookup through Solr:
## Authority Control must be set. Works for tag type inputs where no vocabulary is specified.
choices.plugin.local.subject.personalName = LocalSubjectName
choices.presentation.local.subject.personalName = suggest
authority.controlled.local.subject.personalName = true
choices.order.local.subject.personalName = count
choices.index.local.subject.personalName = personalName_prefix

Then in my discovery.xml file, I created a search filter:

    <bean id="searchFilterSubjectPersonalName" class="org.dspace.discovery.configuration.DiscoverySearchFilterFacet">
        <property name="indexFieldName" value="personalName"/>
        <property name="metadataFields">
            <list>
                <value>local.subject.personalName</value>
            </list>
        </property>
        <property name="facetLimit" value="5"/>
        <property name="sortOrderSidebar" value="COUNT"/>
        <property name="sortOrderFilterPage" value="COUNT"/>

    </bean>

and added this bean in the defaultConfiguration bean such that:

        <property name="searchFilters">
            <list>
                <ref bean="searchFilterTitle" />
                <ref bean="searchFilterAuthor" />
                <ref bean="searchFilterSubject" />
                <ref bean="searchFilterIssued" />
                <ref bean="searchFilterSubjectPersonalName" />
…

Note the "indexFieldName" value="personalName" and choices.index.local.subject.personalName = personalName_prefix Then I restarted Tomcat and performed a reindex, i.e., dspace index-discovery -b. Below is from my local instance:

Thanks @tdonohue,

We could port our implementation to be working on the main (DSpace 9.0) branch. I am reviewing again what could be missing from the "Instruction for reviewers" to help you and other testers to test/review our work.

Thank you. Kind regards, Zoltán Kanász-Nagy

@kanasznagyzoltan : Unfortunately, this has missed our "Feature PR Merge Deadline" of Friday, March 28. As it is a new feature, we are no longer able to add it into DSpace 9.0. It will have to be rescheduled for 10.0 because feedback was not addressed in time for us to re-review it prior to that merge deadline. You are still welcome & encouraged to update this PR appropriately. But, as you can see from the comments above (starting Feb 6), our testers/reviewers had difficulty getting this feature working properly. So, you may also want to verify that the instructions for testing this PR are accurate & no other changes on the backend are necessary (and if changes are necessary on the backend, those should either be documented or put into a related backend PR).

Hi @kanasznagyzoltan , I just wanted to note that I have been working on a similar feature which uses the solr suggest request handler to allow autocomplete in various methods (flat file, existing search terms etc), though it doesn't use ChoiceAuthority. I'll share my work when it is closer to being ready -- it may be that there is no real crossover or incompatibility, or it may be that we can combine our efforts in some way. Thanks for the work so far, it looks good!

To follow up, here are my draft PRs that use the request handler https://github.com/DSpace/DSpace/pull/10855 https://github.com/DSpace/dspace-angular/pull/4439

I actually didn't end up using the ChoiceAuthority impl in my prototype iirc (I need to update my PRs and check work) but I wonder if there is a possibility where we have a unified way of letting users decide whether to autocomplete on:

• Configured facet values via ChoiceAuthority (your solution here)
• Terms of field via Solr Suggest
• Terms from txt file via Solr Suggest
• (also merge in the existing ControlledVocab XML?)

Dear @tdonohue

We have updated our PR merging the dspace-8_x branch to our feature branch and fixed the missing javadoc problem.

@kanasznagyzoltan : Please see the comments above from @kshepherd . We need your teams feedback on how to move this forward alongside @kshepherd 's autocomplete work. Currently, your two PRs are overlapping, so we need to determine a way forward together.

Hi @tdonohue

Our PR is based on the existing choiceAuthority system, and we’ve implemented it as a LocalMetadataLookup that can be enabled with minimal configuration. It requires no frontend changes and is immediately usable even in existing DSpace 8.x installations.

In contrast, Kevin’s proposed solution is a larger change aimed at DSpace 10+ (main branch), and while we support its long-term value, it’s not yet available or stable for current production users.

Because of this, we’d really like to see our simpler solution included in the 8.x branch. This allows institutions (including ours) to benefit from it immediately, and it also allows us to properly account for the community contribution (we’ve invested real time and effort into this feature).

Rather than forcing a decision between the two approaches, we propose letting both coexist — our solution in 8.x for current use, and Kevin’s more comprehensive solution for future releases (DSpace 10+).

We’re of course happy to adapt to any feedback to make this work well alongside Kevin’s direction.

@kanasznagyzoltan : To clarify, because this is a "new feature", we won't be able to release this in an 8.x release. DSpace only allows new features in major releases (x.0 releases). So, the next release that allows for new features is 10.0. The 7.x, 8.x and 9.x branches are only accepting bug fixes, security fixes and accessibility fixes.

So, that's why I was asking about how we can work together with @kshepherd (Kim) to determine a way to move these forward together or work on a single solution together. Having duplicative features in 10.0 is not ideal, obviously, as it would confuse users.

A few other notes:

• I had overlooked that this PR is still against the dspace-8_x branch. It really should be against main because it's a new feature. This was mentioned in one of my comments above.
• I've noticed that prior feedback about how to test this PR has not yet been resolved or answered in comments. If you look above, both myself and another developer had issues getting this to work. I think we still require more details on how to get this working properly, as the instructions in the PR description don't seem to work.

@tdonohue You're absolutely right — this PR should target the main branch, as it's a new feature. I’ll rebase and update it accordingly. I’d also be happy to collaborate with @kshepherd to align our efforts and avoid duplicating functionality in 10.0. Thanks again for the guidance — I really appreciate your time!