evidence icon indicating copy to clipboard operation
evidence copied to clipboard
verified publisher icon evidence-dev

[Bug]: Cannot connect to bigquery using keyfile

Open archiewood opened this issue 1 year ago • 1 comments

Describe the bug

When attempting to add a new connection to bigquery, error is thrown

Cannot convert undefined or null to object

Image

Steps to Reproduce

  1. Create project
  2. Navigate to settings page
  3. Add a new source, bigquery
  4. Add a keyfile
  5. Attempt to Confirm Changes

Logs

Uncaught error while server responding {
  error: Error: ENOENT: no such file or directory, open '/Users/archie/Projects/nyc-taxi/.evidence/template/src/pages/settings/+page.md'
      at async open (node:internal/fs/promises:639:25)
      at async Object.readFile (node:internal/fs/promises:1242:14)
      at async GET (/Users/archie/Projects/nyc-taxi/.evidence/template/src/pages/api/[...route]/evidencemeta.json/+server.js:38:18)
      at async Module.render_endpoint (/Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:43:18)
      at async resolve (/Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/runtime/server/respond.js:443:17)
      at async Module.respond (/Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/runtime/server/respond.js:319:20)
      at async file:///Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:524:22 {
    errno: -2,
    code: 'ENOENT',
    syscall: 'open',
    path: '/Users/archie/Projects/nyc-taxi/.evidence/template/src/pages/settings/+page.md'
  },
  event: {
    cookies: {
      get: [Function: get],
      getAll: [Function: getAll],
      set: [Function: set],
      delete: [Function: delete],
      serialize: [Function: serialize]
    },
    fetch: [Function (anonymous)],
    getClientAddress: [Function: getClientAddress],
    locals: {},
    params: { route: '/settings' },
    platform: undefined,
    request: Request {
      method: 'GET',
      url: 'http://localhost:3000/api//settings/evidencemeta.json',
      headers: Headers {
        host: 'localhost:3000',
        connection: 'keep-alive',
        'sec-ch-ua-platform': '"macOS"',
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36',
        'sec-ch-ua': '"Chromium";v="131", "Not_A Brand";v="24"',
        dnt: '1',
        'sec-ch-ua-mobile': '?0',
        accept: '*/*',
        'sec-fetch-site': 'same-origin',
        'sec-fetch-mode': 'cors',
        'sec-fetch-dest': 'empty',
        referer: 'http://localhost:3000/settings/',
        'accept-encoding': 'gzip, deflate, br, zstd',
        'accept-language': 'en-US,en;q=0.9',
        cookie: 'authjs.callback-url=http%3A%2F%2Flocalhost%3A5173%2F; __stripe_mid=f431d96f-3d80-4958-a831-ffcfcf478caae0c881; _dd_s=logs=1&id=b4ef62cd-3110-4f58-bac4-8894935439d6&created=1732894201698&expire=1732906494625'
      },
      destination: '',
      referrer: 'about:client',
      referrerPolicy: '',
      mode: 'cors',
      credentials: 'same-origin',
      cache: 'default',
      redirect: 'follow',
      integrity: '',
      keepalive: false,
      isReloadNavigation: false,
      isHistoryNavigation: false,
      signal: AbortSignal { aborted: false }
    },
    route: { id: '/api/[...route]/evidencemeta.json' },
    setHeaders: [Function: setHeaders],
    url: URL {
      href: 'http://localhost:3000/api//settings/evidencemeta.json',
      origin: 'http://localhost:3000',
      protocol: 'http:',
      username: '',
      password: '',
      host: 'localhost:3000',
      hostname: 'localhost',
      port: '3000',
      pathname: '/api//settings/evidencemeta.json',
      search: '',
      searchParams: URLSearchParams {},
      hash: ''
    },
    isDataRequest: false,
    isSubRequest: false
  },
  status: 500,
  message: 'Internal Error'
}
Source @ ../../sources/.DS_Store is not a directory!
Uncaught error while server responding {
  error: Error: ENOENT: no such file or directory, open '/Users/archie/Projects/nyc-taxi/.evidence/template/src/pages/settings/+page.md'
      at async open (node:internal/fs/promises:639:25)
      at async Object.readFile (node:internal/fs/promises:1242:14)
      at async GET (/Users/archie/Projects/nyc-taxi/.evidence/template/src/pages/api/[...route]/evidencemeta.json/+server.js:38:18)
      at async Module.render_endpoint (/Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:43:18)
      at async resolve (/Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/runtime/server/respond.js:443:17)
      at async Module.respond (/Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/runtime/server/respond.js:319:20)
      at async file:///Users/archie/Projects/nyc-taxi/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:524:22 {
    errno: -2,
    code: 'ENOENT',
    syscall: 'open',
    path: '/Users/archie/Projects/nyc-taxi/.evidence/template/src/pages/settings/+page.md'
  },
  event: {
    cookies: {
      get: [Function: get],
      getAll: [Function: getAll],
      set: [Function: set],
      delete: [Function: delete],
      serialize: [Function: serialize]
    },
    fetch: [Function (anonymous)],
    getClientAddress: [Function: getClientAddress],
    locals: {},
    params: { route: '/settings' },
    platform: undefined,
    request: Request {
      method: 'GET',
      url: 'http://localhost:3000/api//settings/evidencemeta.json',
      headers: Headers {
        host: 'localhost:3000',
        connection: 'keep-alive',
        'sec-ch-ua-platform': '"macOS"',
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36',
        'sec-ch-ua': '"Chromium";v="131", "Not_A Brand";v="24"',
        dnt: '1',
        'sec-ch-ua-mobile': '?0',
        accept: '*/*',
        'sec-fetch-site': 'same-origin',
        'sec-fetch-mode': 'cors',
        'sec-fetch-dest': 'empty',
        referer: 'http://localhost:3000/',
        'accept-encoding': 'gzip, deflate, br, zstd',
        'accept-language': 'en-US,en;q=0.9',
        cookie: 'authjs.callback-url=http%3A%2F%2Flocalhost%3A5173%2F; __stripe_mid=f431d96f-3d80-4958-a831-ffcfcf478caae0c881; _dd_s=logs=1&id=b4ef62cd-3110-4f58-bac4-8894935439d6&created=1732894201698&expire=1732906506215'
      },
      destination: '',
      referrer: 'about:client',
      referrerPolicy: '',
      mode: 'cors',
      credentials: 'same-origin',
      cache: 'default',
      redirect: 'follow',
      integrity: '',
      keepalive: false,
      isReloadNavigation: false,
      isHistoryNavigation: false,
      signal: AbortSignal { aborted: false }
    },
    route: { id: '/api/[...route]/evidencemeta.json' },
    setHeaders: [Function: setHeaders],
    url: URL {
      href: 'http://localhost:3000/api//settings/evidencemeta.json',
      origin: 'http://localhost:3000',
      protocol: 'http:',
      username: '',
      password: '',
      host: 'localhost:3000',
      hostname: 'localhost',
      port: '3000',
      pathname: '/api//settings/evidencemeta.json',
      search: '',
      searchParams: URLSearchParams {},
      hash: ''
    },
    isDataRequest: false,
    isSubRequest: false
  },
  status: 500,
  message: 'Internal Error'
}

Severity

blocking all usage of Evidence

archiewood avatar Nov 29 '24 18:11 archiewood

Workarounds

Type in the creds directly to connection.yaml and connection.options.yaml

sources/taxi/connection.yaml

# This file was automatically generated
name: taxi # or whatever you want your source to be called
type: bigquery
options:
  authenticator: service-account
  location: US # or other region, based on your account eg EU

sources/taxi/connection.options.yaml Note that the values in this file must be base64 encoded. This prevents issues parsing newline characters, for example in the private_key. You can do this using a variety of online or local software programs.

# This file was automatically generated
# Do not edit this file manually; instead use npx evidence-sdk sources edit

# It should *not* be source controlled, as it likely contain credentials or other sensitive configuration values.
# Values in this file are base64 encoded; https://it-tools.tech/base64-string-converter has an excellent encoder / decoder tool.
# Base64 is NOT encryption, and should not be treated as secure
project_id: YW...[REDACTED]
client_email: ZX...[REDACTED]
private_key: LS0[REDACTED]

archiewood avatar Jan 14 '25 14:01 archiewood