CI function do not behave the same as local

[riderx]

Describe the bug All functions test pass 100% of the time in local but often fail in CI/CD env.

*To Reproduce Steps to reproduce the behavior: In this repo: https://github.com/Cap-go/capgo In local, if I do :

• supabase start
• npm run minio:start && npm run minio:seed ( I use minio i didn't migrate to s3 api of supabase yet)
• supabase functions serve
• deno test --allow-all supabase/functions/tests/* --env=supabase/.env.test --import-map=supabase/functions/import_map.json --fail-fast
• All test pass.
• In github action

name: Run tests

on:
  push:
    branches:
      - 'main'

jobs:
  test_base_sql:
    runs-on: ubuntu-latest
    name: "Run test"
    steps:
      - name: "Checkout capgo"
        uses: actions/checkout@v4
      - name: Install Supabase CLI
        uses: supabase/setup-cli@v1
        with:
          version: 1.176.10
      - name: Set env PROD
        run: echo "SUPABASE_DB_PASSWORD=${{ secrets.SUPABASE_DB_PASS }}" >> $GITHUB_ENV    
      - name: 🍜 Prepare Supabase PROD
        run: supabase link --project-ref xvwzpoazmxkqosrdewyv
        env:
          SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_TOKEN }}
      - name: Run Supabase Start
        run: ENV=local supabase start
      - run: supabase test db
      - uses: denoland/setup-deno@v1
        with:
          deno-version: v1.x
      - name: "Start minio"
        run: npm run minio:start && npm run minio:seed
      - run: deno test --allow-all supabase/functions/tests/* --env=supabase/.env.test --import-map=supabase/functions/import_map.json

Both env run the same way and yet the CI/CD fail, you can see the history of github actions, some endpoint timeout weirdly

Expected behavior Localhost should fail the same way as CI/CD

[riderx]

I think i found my issue it was not exactly the same env

[riderx]

I had sadly to re-open it, it still happening. Example not passing : https://github.com/Cap-go/capgo/actions/runs/9909169516 And passing: https://github.com/Cap-go/capgo/actions/runs/9905586297 No change in the dependency used in edge functions

[sweatybridge]

Let me move this to functions repo who might have better knowledge of deno testing.

@nyannyacha 502 seems like a server side error from edge runtime. Are there any concurrency caveats with running edge runtime locally?

[nyannyacha]

I can see from the CI's logs that the status code was 502. However, this doesn't tell us much.

To determine the exact cause, it's a good idea to have the body returned by the edge runtime endpoint also be printed to the log.

Run deno test --allow-all supabase/functions/tests/* --env=supabase/.env.test --import-map=supabase/functions/import_map.json

I'm guessing from the flags given to the deno test that all test files and tests will be run sequentially.

502 seems like a server side error from edge runtime. Are there any concurrency caveats with running edge runtime locally?

I don't think I can advise anything at this point, as this command evaluates all tests sequentially as far as I know, so there's nothing to comment on for concurrency caveats in the edge runtime. 😓

[riderx]

I managed to resolve the issue, but later again I had once more the same kind of problem. I realized that will come over and over and what is missing is to access in CI to Edge logs : https://github.com/supabase/cli/issues/3216

This will allow us to debug ourself this kind of issue.