Advanced Search can be unintuitive

[alran]

Expected Behavior or Use Case

We have had many of our users comment that the advanced search functionality is unintuitive to them. Here are the types of things our users have commented on:

• functionality related to excluding some results (like this NOT that)
• the search bar doesn't allow you to press down in order to select options/suggestions
• when you initiate a search from the homepage and then hit enter, you can't use the back button to return to the homepage
• "search is unintuitive. the search filters on the side, for example, seem to expand the filtering sometimes?"

Other companies have also asked for feedback from their users:

• ING
  • Parisa@ presented her user research on Amundsen (Slides, recording)
• REA Group is doing user testing with their Alpha users
  • some of their users "struggle with the advanced search features"
  • they might improve the tooltip so that it doesn't say 'regex' since not everyone knows what that means.
  • One user in their testing suggested pre-populating some of the search bars with examples
  • would be interested to experiment with taking away exact search by default

This has come up in at least one comment in the slack group:

• Tanmay: "I am having a hard time figuring out the advanced search. The usual search works perfectly fine and I am able search through all the metadata in elastic search. When I try using advanced search, by setting the source/table name/tags it returns an empty set of results. I am not sure what is the issue, as my usual search works fine."

Service or Ingestion ETL

Amundsen Search Library

Possible Implementation

I realize this issue is a bit vague, but we have gotten overwhelming feedback that our users want search to change (figuring out what exactly that looks like is a separate issue)

[feng-tao]

@alran hey Alyssa, i know there are a lot more we could do to improve the search. And thanks for documenting the issues. I have an internal doc for search syntax and I think we could copy it over to help clarify a bit. In the mean time, we could brainstorm and improve the search?

[alran]

Yes, I think a brainstorm would be good! Perhaps we could get some other people in the community onboard with a brainstorming session as well

[jornh]

Happy to participate as well. What format do you suggest? Can we attract more people to discuss here on this issue or is a small online session more productive?

[alran]

I'm going back to some people internally to try to gather more specific notes that could be used in a brainstorming session. I think it might be nice to have a small online session (I know there is at least one person on my team who'd like to demo a style of search that he thinks is more intuitive). But I also want people to comment here with suggestions and ideas too!

[jornh]

Sounds great! In particular the “use pre-existing” part. Would be great to not have to reinvent this - both on UX level - but in particular if it was already backed by a library. Any product name to drop on this perhaps? To me the JQL advanced bar in JIRA comes to mind with “intellisense” like suggestions - or something like what’s already there with the IU for adding tags in Amundsen.

To kick off a discussion I think we can separate this into two buckets:

1. “Low fruit” usability nits with a pretty obvious resolution with low risk for side effects like some of the ones I list below.
2. Others which require more pro/con discussions, maybe even ultimately writing an RFC.

Low fruit 🍉 🍏 🍌 list:

the search bar doesn't allow you to press down in order to select options/suggestions

=> implement an arrow key keyboard handler to cycle down/up through the list of suggested results.

they might improve the tooltip so that it doesn't say 'regex' since not everyone knows what that means.

Agree, and ‘regex’ is actually the wrong word for how this works - it’s merely just ‘wildcard using *’ which I guess is understandable by more people. Plus it might be possible to add a local help link to either the docco Tao can carve out - or just a slightly improved version of the already existing examples under point 7. and 8. in https://www.amundsen.io/amundsen/installation/#bootstrap-a-default-version-of-amundsen-using-docker

[cleber0120]

A few additional comments:

• Clearing the form is unintuitive: clearing an advanced search term on the side bar by deleting the text from the box doesn't always clear it from the URL. You must select "Clear" in order for the term to be removed.
• The Lyft data model (database -> cluster -> schema -> table) doesn't map exactly to all companies' data models. As a result, it isn't always clear what type of data you're searching for using each of these fields. Since the project is open source, implementers can update the code to include only which fields they want and update the tooltip/name/etc. to something meaningful to them on the advanced search page, but it would be simpler if this was configurable.

I'm interested in participating in a brainstorming session if one is set up!

[sterlins]

@feng-tao I've been trying to set up search doco as well so it'd be great to see what you've got!

I'd also been thinking that highlighting the matching search terms, a la google search results, would help make it more transparent to users why certain tables show higher in results than others:

I might also test whether we can introduce some of the more specific ways to search on the home page, for example:

[feng-tao]

@alran https://github.com/amundsen-io/amundsen/pull/773/files for search tutorial doc

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.