sylph
sylph copied to clipboard
mmcc007
Sylph throws exception when fetching aws devices.
I'm have been trying out the tool but are running into an issue that prevents us from going further. I'm not 100% sure that this is related to the tool or this is a configuration issue but I definitely need some help.
So I've been able to install and run the sylph tool. It is also picking up my sylph.yaml configuration file. The connection to our aws account seems to work as well. The exception I'm running into is the following:
Starting Sylph run 'sylph run 2020-03-11 18:38:05.936' on AWS Device Farm ...
Config file: sylph.yaml
Unhandled exception:
FormatException: Missing expected digit (at character 2)
---------------------------------------------------------------------------...
^
#0 _ChunkedJsonParser.fail (dart:convert-patch/convert_patch.dart:1394:5)
#1 _ChunkedJsonParser.parseNumber (dart:convert-patch/convert_patch.dart:1258:9)
#2 _ChunkedJsonParser.parse (dart:convert-patch/convert_patch.dart:926:22)
#3 _parseJson (dart:convert-patch/convert_patch.dart:31:10)
#4 JsonDecoder.convert (dart:convert/json.dart:495:36)
#5 JsonCodec.decode (dart:convert/json.dart:153:41)
#6 jsonDecode (dart:convert/json.dart:96:10)
#7 deviceFarmCmd (package:sylph/src/base/utils.dart:150:10)
#8 getDeviceFarmDevices (package:sylph/src/device_farm.dart:290:30)
#9 isValidConfig (package:sylph/src/validator.dart:15:25)
#10 sylphRun (package:sylph/src/sylph_run.dart:44:8)
#11 run (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:97:35)
<asynchronous suspension>
#12 main.<anonymous closure> (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:62:13)
#13 AppContext.run.<anonymous closure> (package:tool_base/src/base/context.dart:154:29)
#14 _rootRun (dart:async/zone.dart:1126:13)
#15 _CustomZone.run (dart:async/zone.dart:1023:19)
#16 _runZoned (dart:async/zone.dart:1518:10)
#17 runZoned (dart:async/zone.dart:1465:12)
#18 AppContext.run (package:tool_base/src/base/context.dart:153:18)
#19 runInContext (package:sylph/src/context_runner.dart:11:24)
#20 main (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:61:11)
#21 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#22 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
By the looks of it it is failing when trying to fetch the aws devices. I've actually put a proxy on my Mac to see what the response was to the http call to fetch the aws-devices and the response seems to be pretty normal.
my sylph.yaml file:
# Config file for Flutter tests on real device pools.
# Auto-creates projects and device pools if needed.
# Configures android and ios test runs.
# Builds app, uploads and runs tests.
# Then monitors tests, returns final pass/fail result and downloads artifacts.
# Note: assumes the 'aws' command line utility is logged-in.
# Note: to build the debug iOS app, certain environment variables are required.
# sylph config
tmp_dir: ./tmp/sylph
artifacts_dir: ./tmp/sylph_artifacts
# local timeout per device farm run
# sylph_timeout: 720 # seconds approx
# run on ios and android pools concurrently (for faster results)
concurrent_runs: true
# device farm config
project_name: Test_project
default_job_timeout: 10 # minutes, set at project creation
device_pools:
- pool_name: Google Pixel 2
pool_type: android
devices:
- arn: arn:aws:devicefarm:us-west-2::device:58D6FB12B3624256AED26D0F940D4427
- pool_name: iPhone11
pool_type: ios
devices:
- name: Apple iPhone11
model: iPhone11
os: 13.1.3
test_suites:
- test_suite: example tests 1
main: test_driver/bdd.dart
tests:
- test_driver/bdd_test.dart
pool_names:
- Google Pixel 2
# - iPhone11
job_timeout: 15 # minutes, set per job, over-rides default job timeout above
the arn code is actually in the response when the tool is calling the devices.
Any help will be greatly appreciated.
A device is of the form:
- name: Apple iPhone11
model: iPhone11
os: 13.1.3
not
- arn: arn:aws:devicefarm:us-west-2::device:58D6FB12B3624256AED26D0F940D4427
For example, to find available android Pixel devices:
sylph -d android | grep Pixel
name:Google Pixel, model:Pixel, os:7.1.2, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 2, model:Google Pixel 2, os:8.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 2, model:Google Pixel 2, os:8.1.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 2, model:Google Pixel 2, os:9.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 2 XL, model:Google Pixel 2 XL, os:8.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 2 XL, model:Google Pixel 2 XL, os:9.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 3, model:Pixel 3, os:9.0.0, deviceType:android, formFactor:phone, availability:BUSY
name:Google Pixel 3, model:Pixel 3, os:10.0.0, deviceType:android, formFactor:phone, availability:AVAILABLE
name:Google Pixel 3 XL, model:Pixel 3 XL, os:9.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 3 XL, model:Pixel 3 XL, os:10.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 4 (Unlocked), model:{GA01188-US}, os:10.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel 4 XL (Unlocked), model:{GA01182-US}, os:10.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
name:Google Pixel XL, model:Pixel XL, os:8.0.0, deviceType:android, formFactor:phone, availability:HIGHLY_AVAILABLE
Select the Pixel devices you want to run tests on by name, model, and os and add to an android pool.
@mmcc007 I have tried this form as well :
name: Google Pixel 2
model: Google Pixel 2
platform: ANDROID
os: 9
Have got the same exception.
This command doesn't work for me as well : sylph -d android | grep Pixel
sylph -d android | grep Pixel
Unhandled exception:
FormatException: Missing expected digit (at character 2)
---------------------------------------------------------------------------...
^
#0 _ChunkedJsonParser.fail (dart:convert-patch/convert_patch.dart:1394:5)
#1 _ChunkedJsonParser.parseNumber (dart:convert-patch/convert_patch.dart:1258:9)
#2 _ChunkedJsonParser.parse (dart:convert-patch/convert_patch.dart:926:22)
#3 _parseJson (dart:convert-patch/convert_patch.dart:31:10)
#4 JsonDecoder.convert (dart:convert/json.dart:495:36)
#5 JsonCodec.decode (dart:convert/json.dart:153:41)
#6 jsonDecode (dart:convert/json.dart:96:10)
#7 deviceFarmCmd (package:sylph/src/base/utils.dart:150:10)
#8 getDeviceFarmDevices (package:sylph/src/device_farm.dart:290:30)
#9 getDeviceFarmDevicesByType (package:sylph/src/device_farm.dart:283:10)
#10 run (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:76:32)
#11 main.<anonymous closure> (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:62:13)
#12 AppContext.run.<anonymous closure> (package:tool_base/src/base/context.dart:154:29)
#13 _rootRun (dart:async/zone.dart:1126:13)
#14 _CustomZone.run (dart:async/zone.dart:1023:19)
#15 _runZoned (dart:async/zone.dart:1518:10)
#16 runZoned (dart:async/zone.dart:1465:12)
#17 AppContext.run (package:tool_base/src/base/context.dart:153:18)
#18 runInContext (package:sylph/src/context_runner.dart:11:24)
#19 main (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:61:11)
#20 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#21 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
Confirm that your aws command is configured to return json by default eg,
cat ~/.aws/config
[default]
output = json
region = us-west-2
(https://github.com/mmcc007/sylph#aws-cli-credentials)
A command such as aws devicefarm list-devices should return valid json.
@mmcc007 thanks you for your help, it was configured to return table. Have change it to json, but now i'm running into another issue. Maybe you can give me some advice?
The exception I'm running into is the following:
Starting Sylph run 'sylph run 2020-03-18 20:34:05.781' on AWS Device Farm ... Config file: sylph.yaml Unpacking sylph resources to ./tmp/sylph Creating test bundle for upload... Unhandled exception: Invalid argument(s): Cannot copy from . to ./tmp/sylph/test_bundle/flutter_app #0 _doNothing (package:sylph/src/base/copy_path.dart:15:5) #1 copyPathSync (package:sylph/src/base/copy_path.dart:56:7) #2 LocalPackageManager.copy (package:sylph/src/base/local_packages.dart:104:7) #3 bundleFlutterTests (package:sylph/src/bundle.dart:31:23) #4 sylphRun (package:sylph/src/sylph_run.dart:59:3) <asynchronous suspension> #5 run (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:97:35) <asynchronous suspension> #6 main.<anonymous closure> (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:62:13) #7 AppContext.run.<anonymous closure> (package:tool_base/src/base/context.dart:154:29) #8 _rootRun (dart:async/zone.dart:1126:13) #9 _CustomZone.run (dart:async/zone.dart:1023:19) #10 _runZoned (dart:async/zone.dart:1518:10) #11 runZoned (dart:async/zone.dart:1465:12) #12 AppContext.run (package:tool_base/src/base/context.dart:153:18) #13 runInContext (package:sylph/src/context_runner.dart:11:24) #14 main (file:///Users/angela/flutter/.pub-cache/hosted/pub.dartlang.org/sylph-0.7.2/bin/main.dart:61:11) #15 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32) #16 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
@AnzhelaAninian Looks like the code to handle local packages (needed when local packages are present) expects the path to tmp directory to be absolute:
tmp_dir: ./tmp/sylph
should be
tmp_dir: /tmp/sylph
(I've opened an issue to remove tmp_dir from config file... and another one to remove dependency on aws to be configged for json. Thanks for your feedback!)