Extend semantic search widget to support more than just ECSO MeasurementTypes

[amoeba]

The Annotation filter in the search catalog is currently hard-coded to search for classes in the MeasurementType sub-tree of ECSO. It can quickly be reconfigured to search in another ontology or ontology sub-tree. This has worked well because as it's let us bootstrap some form of semantic search that is immediately useful to users by constraining scope to a single portion of a single ontology.

However, we're starting to bring in annotations from other ontologies and it's too hard to search for datasets with those annotations using the current interface. We should consider if we can extend this to support more than just a sub-tree of a single ontology. Some ideas of mine include:

• Let the deployment have a configured a set of ontologies that the widget might search from and optionally show browse-able trees for
• Find a way to search just within just the set of existing annotations (by free-text). Searching for datasets by browsing the ECSO#MeasurementType tree isn't that useful until you have annotations all over the tree. It's pretty frustrating to click on a term and get zero results. As an aside: If we could show the number of results you'd get if you searched for a term next to a term in the tree browser that'd be cool
• Let users search annotations using the main search box. We could index property labels and this would support terms from any new ontologies

[amoeba]

I think the next iteration of this could and should work pretty similarly to the current implementation (above). Without changing too much, we could extend the current widget to:

• Have a configurable default ontology or ontology sub-tree to search and browse
• Accept a configurable list of ontologies or ontology sub-trees to let the user search and browse
• Show the search and browse widget for the default ontology or ontology sub-tree by default
• Provide an easy way for the user to select an alternative ontology or ontology sub-tree to work with

This might look something like:

This could be done with by re-using the Bootstrap nav pills component:

[amoeba]

I've got a fully working version of the above prototype done,

And now I want to turn to how this would integrate with the variant of this filter we use in the Portals collection search builder:

We currently have three variants of this filter and this would be a fourth. Ideally, we only have one variant, though I'm not sure that's feasible. If anyone has any good ideas I'm all ears.

[amoeba]

I spent some time tinkering and I think I have something that will cover all cases (turning four into one). The basic idea is that we'd extend the SearchableSelect to support a dropdown where the user can pick which ontology they're working against. We could use this on the main search (DataCatalogView) and in the Portals editor equally well. Since it's a change to the SearchableSelect I'm going to coordinate with @laurenwalker and @robyngit for thoughts.

Full link: https://www.figma.com/file/JtkdJiI8Y454uWpYsUwuvF/Multi-Ontology-Search-Mockups?node-id=0%3A1.

[robyngit]

This looks great and I think it's an excellent idea to turn four views into one.

The SearchableSelect uses popovers/tooltips to show extra information about the selected labels when hovered, e.g.:

It might be helpful to show similar tooltips over the ontology to give more information about each one.

[amoeba]

It might be helpful to show similar tooltips over the ontology to give more information about each one.

That's a great idea, @robyngit. Thanks.

[mbjones]

Great work and interesting designs. I suggest we should discuss this semantic search UI on our Thursday semantics call, particularly with expect to what expectations we are making for search users and their knowledge of ontologies.

[robyngit]

Here is a little audit of where we are currently with this issue:

• We have a WIP version of the annotation filter using multiple ontologies in the DataCatalog in this branch. I tested and it works pretty well, though the branch was created ~1.5 years ago and there are some conflicts with the main branch.
• As Bryce pointed out, if we implement this new TabbedAnnotationFilter view, MetacatUI will have yet another version of an Annotation filter. (We currently use the AnnotationFilterView, the SemanticFilterView, and the ~~AnnotatorView~~ (deprecated)). Ideally, we would roll all of these very similar views into a single configurable view.
• Bryce suggested extending the SearchableSelect view. This has already been done in the AnnotationFilterView, though only when the filter is rendered as a multi-select.
• Making a single view that would work for all contexts where the Annotation filter is currently used presents some challenges. For example, in the query builder, a user can keep adding terms to the input, and those terms are displayed in the input itself. In the DataCatalog search, each time a user enters some text, it should be added to the list of "applied filters" instead. (See https://github.com/NCEAS/metacatui/issues/1693)

Here are some options for completing this issue:

The ideal way

Take the opportunity to combine all of the Annotation views into one. This would take the most time.

The fastest way

Use Bryce's WIP view. This needs more testing and a way to configure the ontologies for the repository. The downside is that you would not be able to configure multiple ontologies in all places, e.g. not in the QueryBuilder.

The fastest way without creating a brand new view

Add the ability to select multiple ontologies to the current AnnotationFilterView. This involves basically moving all of the logic Bryce wrote in the TabbedAnnotationFilter to the AnnotationFilterView. The downside is that the AnnotationFilterView would become larger and more complicated than it already is. Also, because the way the multi-select is rendered is so different from the way the single-select is rendered, there would probably be some duplicate logic in the same view, unless we do a bit of a refactor.

[yvanlebras]

Really of interest! Thank you for this work! Not sure I fully understand operationalization scenarios, but just my 2 cents: It appears to me of interest to index all "terminological resources terms" on metacat/metacatui so we can use a searchable form comparable to the one we can use for "attributs" with autocompletion. So having a way to select search terms using the "tree representation of terms" as proposed by @amoeba Oct 12, 2021 + "just" typing the term and having autocompletion would be very nice IMO.

[robyngit]

I'm working on this feature by first combining the two Annotation views into one. Here is a summary of those two views:

AnnotationFilterView

• uses the SearchableSelectView if in "multiselect" mode
• views that use it:
  • DataCatalogView
  • QueryRuleView
  • SemanticFilterView

SemanticFilterView

• extends FilterView
• uses AnnotationFilterView as sub-view
• views that use it:
  • FilterGroupView

classDiagram
    class AnnotationFilterView{
     << ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ >> 
    }
    class SemanticFilterView{
     << ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ >>
    }
    class DataCatalogView 
    class QueryRuleView 
    class FilterGroupView 
    class FilterView
    class SearchableSelectView

    DataCatalogView --> AnnotationFilterView : embeds
    QueryRuleView --> AnnotationFilterView : embeds
    FilterGroupView --> SemanticFilterView : embeds
    AnnotationFilterView --> SearchableSelectView : embeds
    SemanticFilterView ..|> FilterView : extends
    SemanticFilterView --> AnnotationFilterView : embeds