ecosystem-tests icon indicating copy to clipboard operation
ecosystem-tests copied to clipboard

netlify-github: Not actually testing current deploy, but last deploy, and is fine if that is of same version

Open janpio opened this issue 4 years ago • 2 comments

The logic in https://github.com/prisma/e2e-tests/tree/dev/platforms/netlify-ci effectively only works, if we deploy a new version. If we redeploy the same version, the logic is perfectly fine with a previous deployment of the same Prisma version (as a version comparison is the only thing used to confirm the deployment).

Solutions:

  • Check actual deployment, not the main URL for when the deployment is put into "production"
  • Add something unique to the deployment that is being checked for

janpio avatar Apr 10 '21 21:04 janpio

Might also apply to other tests that deploy to an environment that could have earlier, working deployments.

janpio avatar Apr 10 '21 21:04 janpio

Specialized solution for Netlify would be to use https://docs.netlify.com/configure-builds/environment-variables/#git-metadata - that is known to the script pushing to the Github repo. But a bit complicated, as those are not available in Functions env by default but would have to be injected into the function code on build time.

Of course only works for netlify-ci specifically. More generalized solution might be better.

janpio avatar Apr 10 '21 22:04 janpio