nodejs-bigquery icon indicating copy to clipboard operation
nodejs-bigquery copied to clipboard

API methods use overall connection project instead of object's project

Open MasterOdin opened this issue 3 years ago • 0 comments

Environment details

  • OS: macOS 12.4
  • Node.js version: Node 16.15
  • npm version: 8.5.5
  • @google-cloud/bigquery version: 6.0.0

Steps to reproduce

import { BigQuery, Dataset, Routine, Table } from '@google-cloud/bigquery';
import fs from 'fs';
import path from 'path';

const credentials = JSON.parse(fs.readFileSync(path.join(__dirname, 'credentials.json'), { encoding: 'utf-8' }));

const conn = new BigQuery({
  credentials,
  projectId: 'foo'
});

(async () => {
  const [datasets] = await conn.getDatasets({ projectId: 'bigquery-public-data' });
  const dataset = datasets[0];
  console.log(dataset);
  const [tables] = await dataset.getTables();
  console.log(tables[0]);
})()
  .then(() => {
    process.exit();
  })
  .catch((err) => {
    console.error(err);
  });

This will throw an error when I run getTables with message ApiError: Not found: Dataset foo:austin_311. To get this to work, I need to pass projectId to every method call, be it for fetching tables, routines, metadata, etc. This was surprising to me and coworkers where we had assumed that the methods would use the object's projectId versus the overall client. The console log on the dataset gives me:

Dataset {
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  metadata: {
    kind: 'bigquery#dataset',
    id: 'bigquery-public-data:austin_311',
    datasetReference: { datasetId: 'austin_311', projectId: 'bigquery-public-data' },
    location: 'US'
  },
  baseUrl: '/datasets',
  parent: BigQuery {
    baseUrl: 'https://bigquery.googleapis.com/bigquery/v2',
    apiEndpoint: 'https://bigquery.googleapis.com',
    timeout: undefined,
    globalInterceptors: [],
    interceptors: [ [Object] ],
    packageJson: {
      name: '@google-cloud/bigquery',
      description: 'Google BigQuery Client Library for Node.js',
      version: '6.0.0',
      license: 'Apache-2.0',
      author: 'Google LLC',
      engines: [Object],
      repository: 'googleapis/nodejs-bigquery',
      main: './build/src/index.js',
      types: './build/src/index.d.ts',
      files: [Array],
      keywords: [Array],
      scripts: [Object],
      dependencies: [Object],
      devDependencies: [Object]
    },
    projectId: 'foo',
    projectIdRequired: true,
    providedUserAgent: undefined,
    makeAuthenticatedRequest: [Function: makeAuthenticatedRequest] {
      getCredentials: [Function: bound getCredentials],
      authClient: [GoogleAuth]
    },
    authClient: GoogleAuth {
      checkIsGCE: undefined,
      jsonContent: [Object],
      cachedCredential: [JWT],
      _cachedProjectId: 'sql-app-146706',
      keyFilename: undefined,
      scopes: [Array],
      clientOptions: undefined
    },
    getCredentials: [Function: bound getCredentials],
    location: undefined,
    createQueryStream: [Function (anonymous)],
    getDatasetsStream: [Function (anonymous)],
    getJobsStream: [Function (anonymous)]
  },
  id: 'austin_311',
  createMethod: [Function: createMethod],
  methods: {
    create: true,
    exists: true,
    get: true,
    getMetadata: true,
    setMetadata: true
  },
  interceptors: [ { request: [Function: request] } ],
  pollIntervalMs: undefined,
  projectId: undefined,
  location: 'US',
  bigQuery: BigQuery {
    baseUrl: 'https://bigquery.googleapis.com/bigquery/v2',
    apiEndpoint: 'https://bigquery.googleapis.com',
    timeout: undefined,
    globalInterceptors: [],
    interceptors: [ [Object] ],
    packageJson: {
      name: '@google-cloud/bigquery',
      description: 'Google BigQuery Client Library for Node.js',
      version: '6.0.0',
      license: 'Apache-2.0',
      author: 'Google LLC',
      engines: [Object],
      repository: 'googleapis/nodejs-bigquery',
      main: './build/src/index.js',
      types: './build/src/index.d.ts',
      files: [Array],
      keywords: [Array],
      scripts: [Object],
      dependencies: [Object],
      devDependencies: [Object]
    },
    projectId: 'foo',
    projectIdRequired: true,
    providedUserAgent: undefined,
    makeAuthenticatedRequest: [Function: makeAuthenticatedRequest] {
      getCredentials: [Function: bound getCredentials],
      authClient: [GoogleAuth]
    },
    authClient: GoogleAuth {
      checkIsGCE: undefined,
      jsonContent: [Object],
      cachedCredential: [JWT],
      _cachedProjectId: 'foo',
      keyFilename: undefined,
      scopes: [Array],
      clientOptions: undefined
    },
    getCredentials: [Function: bound getCredentials],
    location: undefined,
    createQueryStream: [Function (anonymous)],
    getDatasetsStream: [Function (anonymous)],
    getJobsStream: [Function (anonymous)]
  },
  getModelsStream: [Function (anonymous)],
  getRoutinesStream: [Function (anonymous)],
  getTablesStream: [Function (anonymous)],
  [Symbol(kCapture)]: false
}

So the correct projectId is recorded as part of dataset.metadata.id and in dataset.metadata.datasetReference.projectId, but I'm guessing that that for the API calls, it uses dataset.bigQuery.projectId or something which is different.

MasterOdin avatar Jul 03 '22 16:07 MasterOdin