[Plugins] Improve plugin tests

[janpio]

Change plugin tests to:

• [ ] test all relevant platforms
• [ ] test all relevant platform versions: https://github.com/apache/cordova/issues/38
• [ ] are structured and built in a similar way
  • [ ] same libraries and versions
  • [ ] package.json npm scripts
  • [ ] CI setup and configuration
• [ ] are faster (= take minimal possible time to run)
• [ ] are more consistent (= not flaky or brittle)
• [ ] are simpler to understand
  • [ ] what runs in Travis, what in Saucelabs?
• [ ] are properly documented

Currently blocked by https://github.com/apache/cordova/issues/28

[janpio]

.travis.yml has this bit:

before_install:
- rm -rf ~/.nvm && git clone https://github.com/creationix/nvm.git ~/.nvm && (cd ~/.nvm && git checkout `git describe --abbrev=0 --tags`) && source ~/.nvm/nvm.sh && nvm install $TRAVIS_NODE_VERSION
- node --version
- if [[ "$PLATFORM" =~ android ]]; then gradle --version; fi
- if [[ "$PLATFORM" =~ ios ]]; then npm install -g ios-deploy; fi
- if [[ "$PLATFORM" =~ android ]]; then echo y | android update sdk -u --filter android-22,android-23,android-24,android-25,android-26,android-27;
  fi
- git clone https://github.com/apache/cordova-paramedic /tmp/paramedic && pushd
  /tmp/paramedic && npm install && popd
- npm install -g cordova

(or similar, as configs can be different)

This should:

1. not have those confusing line breaks
2. get the android-22,android-23,android-24,android-25,android-26,android-27 string from a variable so that it is not that easy to overlook
3. probably something like nvm is available on travis by default?
4. not switch from android, then to ios, then again android

[janpio]

Maybe all this repeating stuff should not be handled in the plugin itself?

[raphinesse]

Travis CI itself uses nvm to manage different Node.js versions: https://travis-ci.org/raphinesse/cordova-lib/jobs/435129609#L423

[janpio]

Probably historical reasons plugins are still doing it this way, but can definitely be changed then.

[janpio]

• Chrome is missing for cordova-plugin-file
• Idea: Plugins just define the supported platforms, rest is done by some other package that has all the saucelabs configuration etc
• iOS/Obj-C testing is only present in some repos:

  - env: TEST_DIR=.
    language: objective-c
    osx_image: xcode7
  - env: TEST_DIR=./tests/ios
    language: objective-c
    osx_image: xcode7

• Xcode versions used across tests are pretty random (and old!)
• Node versions used across tests are also not identical ( most use 4.2, some use others)

[janpio]

There actually are docs: https://github.com/apache/cordova-plugin-inappbrowser/blob/master/CONTRIBUTING.md#running-plugin-tests

[janpio]

https://www.browserstack.com/ maybe?

[timbru31]

I'd like to vote to include a npm audit for the plugin tests 👍
Maybe GitHub Actions could be used for this task, too.

[janpio]

https://medium.com/@distillerytech/painless-travis-ci-cd-for-android-project-8c5bce253c4b shows a better way to accept the licences and move the Android API and build tools version to an env var. More lines of boilerplate, but those stay unchanged on version updates - so I would count that as a win on our side.

[raphinesse]

@janpio Maybe also take a look at what I did for the CI tests of cordova-android. I think I simplified the whole Android SDK setup there too.

[janpio]

Thanks for the pointer, will take a look at the changes done to https://github.com/apache/cordova-android/blob/master/.travis.yml as well.

[janpio]

Work in progress via rollout of https://github.com/apache/cordova-paramedic/blob/master/.travis.yml to plugin repos.