ecosystem-tests icon indicating copy to clipboard operation
ecosystem-tests copied to clipboard

fix(ci): Remove usage of nick-invision/retry

Open janpio opened this issue 2 years ago β€’ 6 comments

Removes the retry action, keeps the timeouts.

janpio avatar Jul 10 '23 19:07 janpio

Seems the following tests ran into concurrency problems:

Netlify-CLI binary:

expected:
***"version":"4.17.0-dev.86","createUser":***"id":"12345","email":"[email protected]","name":"Alice"***,"updateUser":***"id":"12345","email":"[email protected]","name":"Bob"***,"users":***"id":"12345","email":"[email protected]","name":"Bob"***,"deleteManyUsers":***"count":1***,"files":["index-browser.js","index.d.ts","index.js","package.json","query-engine-rhel-openssl-1.0.x","runtime","schema.prisma"]***
but got:
***"version":"4.17.0-dev.86","createUser":***"id":"12345","email":"[email protected]","name":"Alice"***,"updateUser":***"id":"12345","email":"[email protected]","name":"Bob"***,"users":***"id":"12345","email":"[email protected]","name":"Bob"***,"deleteManyUsers":***"count":1***,"files":["index-browser.js","index.d.ts","index.js","libquery_engine-rhel-openssl-1.0.x.so.node","package.json","runtime","schema.prisma"]***

Netlify-Github, library:

  rdata: ***"errorType":"PrismaClientKnownRequestError","errorMessage":"\nInvalid `client.user.create()` invocation in\n/var/task/functions/index.js:10:40\n\n   7 \n   8 const id = '12345'\n   9 \n→ 10 const createUser = await client.user.create(\nUnique constraint failed on the fields: (`id`)","trace":["PrismaClientKnownRequestError: ","Invalid `client.user.create()` invocation in","/var/task/functions/index.js:10:40","","   7 ","   8 const id = '12345'","   9 ","→ 10 const createUser = await client.user.create(","Unique constraint failed on the fields: (`id`)","    at zr.handleRequestError (/var/task/node_modules/@prisma/client/runtime/library.js:122:8308)","    at zr.handleAndLogRequestError (/var/task/node_modules/@prisma/client/runtime/library.js:122:7697)","    at zr.request (/var/task/node_modules/@prisma/client/runtime/library.js:122:7307)","    at async Runtime.exports.handler (/var/task/functions/index.js:10:22)"]***
  
  ...
  
  expected:
***"version":"4.17.0-dev.86","createUser":***"id":"12345","email":"[email protected]","name":"Alice"***,"updateUser":***"id":"12345","email":"[email protected]","name":"Bob"***,"users":***"id":"12345","email":"[email protected]","name":"Bob"***,"deleteManyUsers":***"count":1***,"files":["deno","edge.d.ts","edge.js","index-browser.js","index.d.ts","index.js","libquery_engine-rhel-openssl-1.0.x.so.node","package.json","schema.prisma"]***
but got:
***"version":"4.17.0-dev.86","createUser":***"id":"12345","email":"[email protected]","name":"Alice"***,"updateUser":***"id":"12345","email":"[email protected]","name":"Bob"***,"users":***"id":"12345","email":"[email protected]","name":"Bob"***,"deleteManyUsers":***"count":0***,"files":["deno","edge.d.ts","edge.js","index-browser.js","index.d.ts","index.js","libquery_engine-rhel-openssl-1.0.x.so.node","package.json","schema.prisma"]***

Cloudflare-workers-module,

✘ [ERROR] Received a malformed response from the API

  <!DOCTYPE html>
  <!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
  <!--[if I... (length = 7254)

  PUT /accounts/***/workers/scripts/e2e-cloudflare-workers-module -> 524 
  
  If you think this is a bug, please open an issue at: https://github.com/cloudflare/workers-sdk/issues/new/choose

janpio avatar Jul 10 '23 19:07 janpio

Just saw that it did not actually run the full test suite, that is not a good sign.

janpio avatar May 16 '24 13:05 janpio

@janpio indeed

Invalid workflow file: .github/workflows/test.yaml#L1501 The workflow is not valid. .github/workflows/test.yaml (Line: 1501, Col: 9): Unexpected value 'timeout_minutes' .github/workflows/test.yaml (Line: 1518, Col: 9): Unexpected value 'timeout_minutes'

https://github.com/prisma/ecosystem-tests/actions/runs/9112289021

From https://github.com/prisma/ecosystem-tests/actions?query=branch%3Ajanpio%2Fno-retry

Jolg42 avatar May 16 '24 13:05 Jolg42

Seems we rely on the retry still: image

janpio avatar May 16 '24 16:05 janpio

@janpio I think that most failures, in general, are caused in deployments which are shared between library and binary testing.

Jolg42 avatar May 17 '24 08:05 Jolg42

We could start by keeping the retries in .github/workflows/test.yaml and remove them in other files maybe.

Jolg42 avatar May 17 '24 08:05 Jolg42