Feature: Adding scrolling for Flutter Web Apps!

[ff-vivek]

Add smooth animated scrolling for Flutter web with speed parameter support

Problem

Flutter web applications did not respond to Maestro's scroll and scrollUntilVisible commands because:

• Flutter web doesn't use native browser scrolling (uses custom scroll containers)
• Previous implementation only used window.scroll() which doesn't work with Flutter's rendering system
• The speed parameter in scrollUntilVisible was ignored for Flutter web apps

Solution

Implemented smooth animated scrolling for Flutter web using native browser wheel events with cubic ease-in-out easing.

Key Changes

1. Smooth Animated Scrolling (maestro-web.js)

   • Added maestro.smoothScrollFlutter() function that uses requestAnimationFrame for 60fps smooth scrolling
   • Implements cubic ease-in-out easing for natural acceleration/deceleration
   • Breaks scroll distance into small incremental wheel events for fluid animation

2. Speed Parameter Support (WebDriver.kt, CdpWebDriver.kt)

   • Added calculateScrollPixels() to map speed values to scroll distances
   • Added calculateAnimationDuration() to adjust animation timing based on speed
   • Speed now controls both scroll distance and animation duration

3. Improved Bounds Accuracy (maestro-web.js)

   • Force layout recalculation in getNodeBounds() for accurate element positions after scrolling
   • Ensures getBoundingClientRect() returns fresh values

Technical Details

Speed to Scroll Distance Mapping:

• speed=100: 3000px in 300ms (very fast)
• speed=80: 2000px in 400ms (fast)
• speed=50: 1000px in 500ms (default)
• speed=20: 400px in 600ms (slow)
• speed=0: 200px in 700ms (precise)

Animation:

• Uses requestAnimationFrame for smooth 60fps rendering
• Cubic ease-in-out easing: eased = progress < 0.5 ? 4*p³ : 1-(-2p+2)³/2
• Dispatches incremental wheel events during animation
• Promise-based to ensure completion before continuing

Flutter Detection:

• Automatically detects Flutter web apps by checking for flutter-view, flt-glass-pane, or flt-renderer elements
• Only applies smooth scrolling to Flutter web (HTML web uses existing window scrolling)

Backward Compatibility

✅ No breaking changes

• HTML web scrolling unchanged (uses standard window.scroll())
• Existing tests continue to work without modification
• Only affects Flutter web applications
• Falls back gracefully if Flutter elements not found

Usage

# Basic scrolling (now works with Flutter web!)
- scroll

# scrollUntilVisible with speed control
- scrollUntilVisible:
    element: "Item 100"
    direction: DOWN
    speed: 80        # Now properly controls scroll speed!
    timeout: 10000

Testing

Tested with:

• Flutter web applications with ListView widgets
• Various speed values (0, 20, 50, 80, 100)
• scrollUntilVisible command
• Hybrid apps (HTML + Flutter web)

Files Changed

• maestro-client/src/main/resources/maestro-web.js (+59 lines)
• maestro-client/src/main/java/maestro/drivers/WebDriver.kt (+40 lines)
• maestro-client/src/main/java/maestro/drivers/CdpWebDriver.kt (+40 lines)

Related Issues

Fixes longstanding issue where scroll and scrollUntilVisible commands did not work on Flutter web applications.

Demo Video

https://github.com/user-attachments/assets/09687e82-df24-4c94-a194-a1fffae68579

---

Checklist

• [x] Code follows project style guidelines
• [x] No linter errors
• [x] Backward compatible (no breaking changes)
• [x] Works with both WebDriver and CdpWebDriver
• [x] Tested with Flutter web applications
• [x] HTML web scrolling remains unchanged

[ff-vivek]

As a Maestro user I was asked to provide feedback. I really love this solution as it makes it possible to be used in Flutter Web without changes to the yaml file. If desired, I could also try it out on my machine. Are there instructions how to install and try out Maestro from a branch?

1. Clone this repo and checkout the PR's branch.
2. Install maestro using a script installLocally.sh
3. Use maestro, like you have ever did using maestro commands keyword

[krille-chan]

As a Maestro user I was asked to provide feedback. I really love this solution as it makes it possible to be used in Flutter Web without changes to the yaml file. If desired, I could also try it out on my machine. Are there instructions how to install and try out Maestro from a branch?

1. Clone this repo and checkout the PR's branch.

2. Install maestro using a script installLocally.sh

3. Use maestro, like you have ever did using `maestro commands` keyword

Thanks, I followed the steps but looks like it is not working for me. So it does scroll but scrollUntilVisible isn't able to find the item by text while finding items by text in general works. To reproduce:

Flutter widget:

ListView.builder(
        itemBuilder: (context, index) => ListTile(title: Text('Item $index')),
),

Maestro test:

url: http://localhost:8081
---
- launchApp
- tapOn: "Item 1" # This can be clicked, so text finder is working
- scrollUntilVisible:
    element:
      text: "Item 30" # Just scrolled by and ignored

[ff-vivek]

As a Maestro user I was asked to provide feedback. I really love this solution as it makes it possible to be used in Flutter Web without changes to the yaml file. If desired, I could also try it out on my machine. Are there instructions how to install and try out Maestro from a branch?

1. Clone this repo and checkout the PR's branch.

2. Install maestro using a script installLocally.sh

3. Use maestro, like you have ever did using `maestro commands` keyword

Thanks, I followed the steps but looks like it is not working for me. So it does scroll but scrollUntilVisible isn't able to find the item by text while finding items by text in general works. To reproduce:

Flutter widget:

ListView.builder(
        itemBuilder: (context, index) => ListTile(title: Text('Item $index')),
),

Maestro test:

url: http://localhost:8081
---
- launchApp
- tapOn: "Item 1" # This can be clicked, so text finder is working
- scrollUntilVisible:
    element:
      text: "Item 30" # Just scrolled by and ignored

Try this : https://github.com/ff-vivek/Maestro_PR_Demo/blob/main/maestro_demo.zip

Please read Readme.md file

[krille-chan]

Try this : https://github.com/ff-vivek/Maestro_PR_Demo/blob/main/maestro_demo.zip

Please read Readme.md file

Unknown Property: flutter-id

[Fishbowler]

@ff-vivek Have you been able to get any feedback on this yet?

[ff-vivek]

Try this : https://github.com/ff-vivek/Maestro_PR_Demo/blob/main/maestro_demo.zip Please read Readme.md file

Unknown Property: flutter-id

Possible because, you have not merged #2824 PR locally. This PR will enable a configuration to access custom ids like flutter-id. e.g.

identifierConfig:
  flt-semantics-identifier: flutter-id

Please make sure you merge both prs to run the example shared.

https://github.com/mobile-dev-inc/Maestro/pull/2828 https://github.com/mobile-dev-inc/Maestro/pull/2824

Alternatively, if you still face this issue,

update script to accept other identifier than flutter-id.

[ff-vivek]

@ff-vivek Have you been able to get any feedback on this yet?

Hi @Fishbowler, I am waiting on @krille-chan to recheck the example with steps given in previous comment.

Are there any comments from your end?

[krille-chan]

Try this : https://github.com/ff-vivek/Maestro_PR_Demo/blob/main/maestro_demo.zip Please read Readme.md file

Unknown Property: flutter-id

Possible because, you have not merged #2824 PR locally. This PR will enable a configuration to access custom ids like flutter-id. e.g.

identifierConfig:
  flt-semantics-identifier: flutter-id

Please make sure you merge both prs to run the example shared.

#2828 #2824

Alternatively, if you still face this issue,

update script to accept other identifier than flutter-id.

it now works like this:

url: http://localhost:8081
identifierConfig:
  flt-semantics-identifier: flutter-id
---
- launchApp
- tapOn: "Item 1" # This can be clicked, so text finder is working
- scrollUntilVisible:
    element:
      flutter-id: "item_30"

With this kind of list:

ListView.builder(
        itemBuilder: (context, index) => ListTile(
          title: Semantics(
            identifier: 'item_$index',
            child: Text('Item $index'),
          ),
        ),
      )

So it scrolls down until it is visible. This way it would work. However it surprises me, that I can just find the ListTile by the Text when not scrolling but when scrolling it requires a Semantics widget, which actually should be obsolet as there is Text 🤔

[ff-vivek]

Try this : https://github.com/ff-vivek/Maestro_PR_Demo/blob/main/maestro_demo.zip Please read Readme.md file

Unknown Property: flutter-id

Possible because, you have not merged #2824 PR locally. This PR will enable a configuration to access custom ids like flutter-id. e.g.

identifierConfig:
  flt-semantics-identifier: flutter-id

Please make sure you merge both prs to run the example shared. #2828 #2824 Alternatively, if you still face this issue, update script to accept other identifier than flutter-id.

it now works like this:

url: http://localhost:8081
identifierConfig:
  flt-semantics-identifier: flutter-id
---
- launchApp
- tapOn: "Item 1" # This can be clicked, so text finder is working
- scrollUntilVisible:
    element:
      flutter-id: "item_30"

With this kind of list:

ListView.builder(
        itemBuilder: (context, index) => ListTile(
          title: Semantics(
            identifier: 'item_$index',
            child: Text('Item $index'),
          ),
        ),
      )

So it scrolls down until it is visible. This way it would work. However it surprises me, that I can just find the ListTile by the Text when not scrolling but when scrolling it requires a Semantics widget, which actually should be obsolet as there is Text 🤔

Perfect that Scrolling worked for you.

Regarding the Semantics requirement for the Text widget, I would suggest you check out the implementation in Flutter Great to hear that the scrolling worked for you.

Regarding the semantics requirement for the Text widget, I recommend reviewing the implementation in the Flutter engine.

@Fishbowler, what are the next steps to get this feature merged?

[luizvnegrini]

perfect, this will solve some of the main issues for flutter web. thanks!

[nikzen]

Hi, what is the current status on the PR? We would like to give maestro another try with flutter but it seems this pr is kind of relevant.

[Fishbowler]

Just waiting on some engineering capacity to get it reviewed. I've had a play with it, and it all LGTM, but I wanted a second set of eyes before hitting the merge button.