Support Regex based Recommendation

[HerrKrishna]

Is your feature request related to a problem? Please describe.

I would like to add a recommender that is able to make recommendations based on regular expressions. In particular i need a recommender that will recommend a user defined annotation that describes a semantic field based on regular expressions.

For example the user could annotate "for 4 weeks" with the semantic field annotation "time period" The user will then be prompted to enter a regex that captures the string that has just been annotated and similar ones, in this example: "for \d+ weeks". From then on the recommender will recommend the annotation "time period" if a token sequence matches the regex "for \d weeks".

Often regex will capture more than the intended sequences. For this case, the recommender should keep count of how many times recommendations based on a certain regex were accepted and rejected. If the ratio of accepted to rejected falls below a certain threshold this regex should no longer be used for recommendations.

Describe the solution you'd like

I have already implemented a solution. It needs some cleaning up before it can be released though. Also in order for the recommender to be able to tell which regexes were accepted/rejected i had to make changes to RecommendationAcceptedEvent and RecommendationRejectedEvent so that they contain information about the regex the recommendations were based on.

So far i handled the user input prompts with message boxes. I realize that this is suboptimal and would like to ask for guidance for alternatives.

Describe alternatives you've considered An external recommender might be an alternative, however then it would be impossible to keep count of how many recommendations were accepted/rejected and based on which regex afaik.

Additional context All this is part of my work for Katharina Jacob at University of Heidelberg, Germanistisches Seminar. https://www.gs.uni-heidelberg.de/sprache02/mitarbeiter/jacob.html

[jcklie]

Hi, this sounds really interesting. How is the performance? We use traits to configure recommender, you can have a look at the stringmatcher gazeteer or external recommender for an example. For the gazeteer, we let users upload a text file and then use that as a dictionary, maybe that is close to what you want. Please open a PR (early is better, then it is easier to change things) and we will have a look.

[jcklie]

You can also create a new sidebar like we have for a search, that might also work.

[HerrKrishna]

I want to add a Gazeteer to my Recommender. In order to do this i've created an AutoConfiguration Class:

@Configuration 
@ConditionalOnBean(RecommendationService.class)
public class RegexRecommenderAutoConfiguration
{

However this doesn't get picked up by Spring. If go into the INCEpTION.java class and comment two lines it works:

@SpringBootApplication
@EnableGlobalMethodSecurity(prePostEnabled = true)
@ComponentScan(
        basePackages = { 
                "de.tudarmstadt.ukp.inception",  
                "de.tudarmstadt.ukp.clarin.webanno" },
        excludeFilters = {
            // @Filter(type = FilterType.REGEX, pattern = ".*AutoConfiguration"),
            @Filter(type = FilterType.CUSTOM, classes = TypeExcludeFilter.class),
            // @Filter(type = FilterType.CUSTOM, classes = AutoConfigurationExcludeFilter.class),
            @Filter(type = FilterType.ASSIGNABLE_TYPE, classes = { `

Why do i need to remove these filters for my own AutoConfiguration to get picked up, but not for the StringMatchingRecommenderAutoConfiguration?

[reckart]

For auto config to work properly, auto configuration classes must be registered via a META-INF/spring.factories file (in src/main/resources). They must not be detected via package scanning (otherwise they would be loaded twice, that is why they are excluded in the INCEpTION.java class).

https://docs.spring.io/spring-boot/docs/2.0.0.M5/reference/html/boot-features-developing-auto-configuration.html

[HerrKrishna]

Thank you. That was very helpfull.

[obtic-sorbonne]

Dear @HerrKrishna How can I use a recommender with regular expressions?

Thank you!

[obtic-sorbonne]

Dear @reckart Is there any way to use regex with recommander? Thanks you.

Dear @HerrKrishna How can I use a recommender with regular expressions?

Thank you!

[reckart]

@obtic-scai At the moment, the easiest would probably be to hook up a custom regex recommender via the external recommender mechanism: https://github.com/inception-project/inception-external-recommender

[obtic-sorbonne]

@reckart Thank you for the information ! best