patrol icon indicating copy to clipboard operation
patrol copied to clipboard

Using forward slash (/) in patrolTest description prevents tests from executing

Open RyanParkGH opened this issue 2 years ago • 3 comments

Steps to reproduce

  1. Create a new patrolTest
  2. For the patrolTest description include a '/' in the String

patrolTest('Testing forward slash / breaking execution', nativeAutomation: true, ($) async { ... }

  1. Run the patrolTest with patrol test -t integration_test/forward_slash_desc_test.dart
  2. Remove the '/' from the description String

patrolTest('Testing forward slash breaking execution', nativeAutomation: true, ($) async { ... }

  1. Run the patrolTest with patrol test -t integration_test/forward_slash_desc_test.dart

Actual results

Test will not execute (even though it says it has) when there's a / (forward slash) included in the patrolTest description. Test will appear as if it is completed, but it won't actually run. Errors in the test will not be caught, no failure results will be returned.

Removing the / (forward slash) fixes the issue and the test executes as expected and returns a proper completion or failure result.

Logs

Logs
<!-- Replace this line with your logs. Do not remove the backticks! -->

Patrol version

patrol: ^2.3.1

Patrol Doctor output

Patrol Doctor output
In next major release, patrolTest method will be intended for UI tests.
If you want to use Patrol in your widget tests, use patrol_finders package.

For more information, see https://patrol.leancode.co/patrol-finders-release
Disable this warning by setting the PATROL_FINDERS environment variable.
      
Patrol CLI version: 2.2.1
Program adb found in /Users/<username>/Library/Android/sdk/platform-tools/adb
Env var $ANDROID_HOME set to /Users/<username>/Library/Android/sdk
Program xcodebuild found in /usr/bin/xcodebuild
Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller

Flutter Doctor output

Flutter Doctor output
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.7.3, on macOS 14.0 23A344 darwin-arm64, locale en-CA)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] IntelliJ IDEA Ultimate Edition (version 2023.1.2)
[✓] Connected device (3 available)
[✓] HTTP Host Availability

• No issues found!

RyanParkGH avatar Oct 30 '23 21:10 RyanParkGH

Yeah, this is not allowed because of what happens inside Android Test Orchestrator. It creates a file for each test case, and if the filename contains a slash, it all falls apart. If you did adb logcat and grepped for Exception or something similar, you'd find the crash.

For now, don't use /.

bartekpacia avatar Nov 03 '23 09:11 bartekpacia

@bartekpacia thank you for the update, that is incredibly strange given it's within a String. IIRC I was looking at logcat within Android Studio and there was no error. But I will check that next time an odd behaviour occurs!

Thanks for the excellent library, it's appreciated!

RyanParkGH avatar Nov 03 '23 17:11 RyanParkGH

package:patrol should check if test name passed to patrolTest() contains invalid characters (such as /) and throw an error early.

bartekpacia avatar Nov 20 '23 12:11 bartekpacia