Forks don't/can't run all the tests

[siddharthkp]

trafficstars

Describe the bug

Forks of the repo can't run all the tests because some of them need API tokens and other secrets to run

This is a security measure by travisCI because you can just read and print these secrets. You can only use these secrets in the main repo (auth0/cosmos).

Problem: Every contribution we get from an outside contributor will have failed tests.

Example: https://travis-ci.org/auth0/cosmos/builds/441308410#L594

Error: You must provide an app code  -- visit https://www.chromaticqa.com to get your code.

These are the useful tokens in our CI environment

1. Chromatic app code - we can probably make this public because our chromatic builds are also public 🤔, not sure if someone can override the master or delete builds etc (@domyen @zol probably can tell us if that's okay, UPDATE: @zol says it's alright)

2. Now API token - this one is needed to deploy to a staging instance, we can't expose this because it gives access to all operations on our zeit/now account

3. Github token for status API - this is what adds the staging url back to the pull request. Even though the permissions for this are limited to adding status URL, I don't feel comfortable exposing it

Expected behavior

We should probably not run tests that we know will fail by adding a check to see which repo is the build running on?

This should be easy to do with https://github.com/siddharthkp/ci-env

Workaround

If as cosmos contributor, you would like to see the tests run, you need to create a new branch in auth0/cosmos and merge the external pull request into it, this will trigger all the tests

You can optionally add the person as a collaborator to auth0/cosmos so that they can continue working on their feature/fix on this new branch (don't worry, only cosmos team members from auth0 can push to master)

[zol]

@siddharthkp it's ok to make the Chromatic app_code public. It can only be used to create new builds -- it's not possible to use it to read data or do any other types of writes.

[siddharthkp]

@zol Thanks!

[siddharthkp]

Seems like moving to the now-github integration will solve the 2nd problem: https://zeit.co/blog/deploy-github-forks

They support aliases as well now: https://zeit.co/blog/every-push-now

[francocorreasosa]

Sounds like a good solution!

[siddharthkp]

1. Have made the chromatic app-code public. Back merging master in your PR will fix that.

2. The now-github integration is not active yet. Have raised a request with the admin folks (Re-opening issue to track that)