User test for tooltip discoverability

[SMR9220]

Dependency

• [x] #1179
• [x] #1223

Overview

We need to conduct a new user test to determine if it is clear to users that there are tooltips on page 1, as well as throughout the application so that we can know if we still need to make changes or if the changes we made solve the problem.

Action Items

• [x] create research plan doc
• [x] Create test support materials
• [ ] add info wiki
• [x] test with users

Resources/Instructions

• Related to #1218
• Tooltip Discoverability User Testing Insights spreadsheet
• [3-18-23] Tooltip Discoverability - Presentation
• UI/UX Tooltip Discovery

[jarmitageux]

Progress: Testing completed, analysis begun Blockers: none Availability: meeting 2x/week and asynchronous work between ETA: for complete report for leads review Nov 5th

[Biuwa]

@Jane4925 Is this complete?

[Noushie]

UXD team has a few propositions for dealing with this issue based on the information UXR presented us- we would love some UXR team input when possible, thanks! @Jane4925

[Biuwa]

• #975

[mojimoh]

@jarmitageux, please provide an update before Wednesday's meeting

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (if necessary): "Add any pictures that will help illustrate what you are working on."

[jarmitageux]

Progress: Presentation deck draft in progress 90% done Blockers: Time available and lots of information to organize Availability: 5 hrs ETA: Draft will be ready for Sat 10am meeting for review and comment I'm available to attend Sat meeting if desired by team.

[ExperimentsInHonesty]

Bonnie's notes from slide presentations Slide numbers and notes

• 3 has too much information
• 5 say something about why we reviewed it with these technically savvy user group
  • new slide add bullets about which features we were testing
• video how to find tooltips - lets talk about it
• 11 could improve via a animated gif
• 19 should be earlier in the deck (in a summary)
• 20 walk them though the chron in date order oldest to newest (17-19) could be tighter
• show image of icon

[jarmitageux]

Progress: Revising slide deck after receiving feedback Blockers: None Availability: 5 hrs ETA: Aiming to have revisions done by end of Feb

Requesting any other feedback on the presentation and/or deck by EOD this Fri, Feb 10th

[Noushie]

UXD Presentation (2/8):

2_8 TDM Design Presentation.pdf

EDIT* Sorry I didn't see previous messages - here is the PDF version

[mojimoh]

Moji's Comments:

1. Create an appendix and move the slides with comments to the appendix
2. Proposed Slide order for Presentation

• Executive summary
• Agenda
• Key Research questions
• Summary of Key Findings
• Recommendations
• Add in Anousha's proposals for tooltip discover (pull 1 slide per proposal, for a total of 3 slides)
• Appendix

3. Create 1 slide with the key research questions listed.
4. Create 1 slide with all the key findings listed

[jarmitageux]

@Noushie Could you please move your presentation deck into the shared drive or link a pdf. We do not have access to the file you linked. Thanks!

[jarmitageux]

Progress: Requested revisions implemented in presentation deck- just finalizing last slide and adding considered approaches to solution Blockers: Final approval by PMs and Bonnie Availability: 5 hrs ETA: Will be ready and available to present at March 21st stakeholder's meeting

[jarmitageux]

REMOVED: * 3 has too much information DONE: * 5 say something about why we reviewed it with these technically savvy user group DONE: * new slide add bullets about which features we were testing NARROWED TO 1 SHORTER VIDEO: * video how to find tooltips - lets talk about it DONE: * 11 could improve via a animated gif DONE: * 19 should be earlier in the deck (in a summary) DONE: * 20 walk them though the chron in date order oldest to newest (17-19) could be tighter DONE: * show image of icon (pointer finger cursor)

[ExperimentsInHonesty]

Notes about the new deck

• slide 3 you mentioned explore icons. what are they?
• slide 7 is blurry. Bring it back in and mind the aspect ratio when re sizing
• slide 11 mention and add text about the fact that the users are internal hack for la members and that in the past their responses have been the similar. Not sure the third interview is necessary
• Slide 15 the second set of videos is better than the first.
• Slide 18 Please change this out for a picture of a confused man or non binary person.
• Slide 22 voice out what the differences were
• several slides: change the yellow highlighting to something more visible

New suggestion

• 14 - Side note, do we have any designs where there is a plain chevron for each item on the right side of the words?

Also, something we never considered doing (was adding a message to the end of the word when they mouse over Currently: hover makes it bold and underline Proposed: Hover make it says Car Share Parking (click to see definition)

[ExperimentsInHonesty]

Some possible images for the slide https://commons.wikimedia.org/wiki/File:Man_Stressed_At_Work.jpg https://commons.wikimedia.org/wiki/File:Man_Overworked_At_Work.jpg https://commons.wikimedia.org/wiki/File:Frustrated_man_at_a_desk_%28cropped%29.jpg https://commons.wikimedia.org/wiki/File:Burnout_At_Work_-_Occupational_Burnout.jpg

[jarmitageux]

Thank you @ExperimentsInHonesty! Those images were super helpful. Could you please review slides 11 (updated video and slide text) and 29 (new slide with next steps).

I think I made all the changes to the deck. Let me know if you see anything else that needs changing.

[Noushie]

Hello,

Truly apologise for the lateness here, I've been pretty swamped lately- I created a new slide show with the updated versions of designs + notes. I can definitely get rid of any wording or tweak it, I also mentioned the plans for some very basic user testing.

The link can be found here as well as a shortcut in the design folder of TDM calculator drive!

[ExperimentsInHonesty]

Please change the first slide to use the same format as the template we use for presenting to stakeholders.

[ExperimentsInHonesty]

@Noushie Thanks for making deck. Ebi and I have reviewed it (about 2 hours worth). Here are changes we made and notes about additional changes needed

General

• [x] Please change the first slide to use the same format as the template we use for presenting to stakeholders.
• Ebi and I added some slides to help make the presentation easier to follow
  • Agenda
  • Section Header Slides

ToolTip Discovery V1: Help Icon (Part 1 of 3)

• I divided this from the sections that follow, while keeping it connected
• I renamed the slide to ToolTip Discovery V1: Help Icon (Part 1 of 4)

ToolTip Discovery V1: Help Icon w/ Pop-up (Part 2 of 3):

• I divided this into two slides to make it easier to understand
  • ToolTip Discovery V1: Help Icon: Popup video (Part 2 of 4)
  • ToolTip Discovery V1: Help Icon: Popup: Text (Part 3 of 4)
    • and made a new mockup which includes the text box image you provided.

ToolTip Discovery V1: Help Icon w/ Tutorial - (Part 3 of 3):

• Renamed the slide to ToolTip Discovery V1: Help Icon w/ Tutorial (Part 4 of 4)

ToolTip Discovery V2: Hover Message:

• The mockup didn't match what is currently on the site (both dev and prod). I have done alternative mock ups (using a screen shot of dev and a text box.
  • I put your old slide on skip (and titled it Mockup 1, so you could see the difference. See also screenshots, with the first 4 lines, here:

Mockup 1 - depreciated

temporary mockup 2

• I can already tell they are going to miss the text. "click to see definition" needs to be more distinctive. Possibly different color text, highlighted, or with an icon(s) in addition to the text. See temporary mockups 3, 4 & 5.
  • [x] 5 will need to be remade (see note)

temporary mockup 3

temporary mockup 4

temporary mockup 5

The highlighted (in green) questions marks should be replaced with the circle version we have in the sidebar

ToolTip Discovery V3 (TENTATIVE): Top Static Box

• I renamed the slide to ToolTip Discovery V3 (TENTATIVE): Top Static Box: Blue
• [x] Make this the first content slide
• [x] Add to the speaker notes: "while we did try this on the strategies page in the past, we did not try placing the box on the first page.
• [x] Create another version of this mockup with the blue question mark changed to green, as it is in the side panel. We are keeping both for the purpose of testing.
  • I created a blank slide for you to use

[Noushie]

Link to the [presentation]

(https://docs.google.com/presentation/d/1Q1N_KHrKIXWDjbSJdqznhBJDP1IMEWOpX9X4rASbgcU/edit?usp=sharing)

**4/11 Update for [Tooltip Discoverability UI/UX Slides]

• Replaced title with TDM deck cover (I named it Tooltip Discoverability (UI/UX) with the projected date of presentation - but I can change it if that's incorrect titling)

• Added question icons for mockup 5

• Replaced all placeholders images with formal mockups

• Added stated speaker note

• Created new green static box (I tried making the clickable terms bolded, green to match - hypothesising the colour might create an association between the presented text and terms/new contrast to the page)

4/17 Update for [Tooltip Discoverability UI/UX Slides]

Replaced the large blue help icon with a green one

Created static box with green icon

Usability test for the main idea (large help icon) is out in UI/UX community

[jarmitageux]

Final report on research:

Presented at the April 4, 2023 stakeholder's meeting. The presentation deck is linked in Resources at the top of this issue page, and will be saved to the TDM Calculator Wiki in UX Research, Usability Testing 3 documentation.

This report included findings from Round 3 testing as well as a thorough research review from the past 2 years (2021-2023) of research findings regarding the discoverability of the information (aka tooltips) that is opened by clicking on text.

Hover state (cursor change to pointer now shown):

Opened:

Summary of Key Findings:

• Written instructions are not effective in helping users discover tooltips.
• Users do not find tooltips that are under text indicated by hover state changes and requiring clicking.
• When looking for information, users are more likely to seek, hover, and click icons.

See presentation deck for full report and recommendations.

[Noushie]

View in figma here

View usability testing results here

![imag

Rationale for designs above:

• Bonnie: Wanted to test w/o the clutter of multiple icons

• Icons only show up upon UI

• 44% of all 18 testers clicked into the input boxes as their initial behaviour (looking at UXR - this heavily aligns with their moderated testing)

• For users that did click the term directly (12/18 users), 58% of them expected some sort of icon ('i' or question mark icon)

• In order to maintain accessibility and a more common use icon in this case - settled on 'more information' icon which majority of web users/even testers recognise as in-line help

• In order to better connect the icon to the term itself (since users tend to be around the input boxes) -> example placeholder text is also recommended (design team also hypotheses this may reduce the confusion around terms to see a valid example as well)

Large question mark

Verdict - it was not very effective placement (most users clicked in the top-fold of the screen and it was not recognised as a way to get help for the specific issue of not knowing a term)

• If tutorials would be added - I hypothesis that having a descriptive text button (e.g., tutorials) placed in the top right or left corner (users were typically hovering around)

[Noushie]

I think also replacing 'placeholder text' w/ actual examples might be very beneficial (as it may give users an inkling as to what things mean before a working definition is shown?)

[Jane4925]

Abby and Melissa are working the google doc version of the wiki page and plan to move it the actual wiki page when ready.

[Biuwa]

Please provide an update:

1. Progress(What is the current status of your project? What have you completed, and what is left to do):
2. Blockers(Difficulties or errors encountered):
3. Availability(How much time will you have this week to work on this issue):
4. ETA(When do you expect this issue to be completed):
5. Pictures (if necessary)(Add any pictures that will help illustrate what you are working on):

[Biuwa]

@melissaperry09, @abbyrivard

Please provide an update:

1. Progress(What is the current status of your project? What have you completed, and what is left to do):
2. Blockers(Difficulties or errors encountered):
3. Availability(How much time will you have this week to work on this issue):
4. ETA(When do you expect this issue to be completed):
5. Pictures (if necessary)(Add any pictures that will help illustrate what you are working on)

[Parisajf]

@melissaperry09 @abbyrivard In the last steakholder prep meeting you mentioned we are still working on this, would you add the remained steps in the comment?

[Parisajf]

Issue #1492 is covering the add info to wiki