cloud-functions-emulator icon indicating copy to clipboard operation
cloud-functions-emulator copied to clipboard

Reading a bucket behaves different on emulator

Open stefanbuck opened this issue 8 years ago • 1 comments

Description

My function behaves different when running in the emulator. I wrote a simple function which reads a bucket file. Running the function on remote prints works as expected and print out the full json file. The same function in the emulator logs just the last ~10 lines from this file.

Environment details

  • OS: MacOS X 10.11.6
  • Node.js version: 6.11.1
  • npm version: 3.10.10
  • google-cloud-node version: Google Cloud SDK 164.0.0 alpha 2017.03.24 beta 2017.03.24 bq 2.0.24 core 2017.07.25 gcloud gsutil 4.27 kubectl

Steps to reproduce

  1. Given we have this cloud function
const gcs = require('@google-cloud/storage')();

exports.myFunc = function myFunc(event, callback) {
  const { bucketId, objectId } = event.data.attributes;

  const bucket = gcs.bucket(bucketId);
  const file = bucket.file(objectId);

  file.download().then((data) => {
    console.log('Content:', data.toString());
    callback();
  }).catch(callback);
};
  1. and a valid json file test.json in our bucket my-bucket
  2. We start the cloud emulator with functions start
  3. and deploy the function with functions deploy receiveMetadata -T=some-topic-e=myFunc
  4. together with functions call receiveMetadata --data='{"attributes":{"bucketId": "my-bucket", "objectId":"test.json"}}'
  5. I expect that console.log prints out the full test.json content, but it only show a subset of the file (last buffer data). A deploy of the same function to the Google Cloud Prints out the full test.json content

stefanbuck avatar Jul 31 '17 13:07 stefanbuck

Hmm, I was unable to reproduce. What version of @google-cloud/storage did you use? I used 1.2.1.

I deployed your function to my emulator, and it successfully downloaded and printed a file that has over 1000 lines of text.

It might possibly work better if you did this instead:

const gcs = require('@google-cloud/storage')();

exports.myFunc = function myFunc(event, callback) {
  const { bucketId, objectId } = event.data.attributes;

  const bucket = gcs.bucket(bucketId);
  const file = bucket.file(objectId);

  file.createReadStream()
    .on('error', callback)
    .on('end', callback)
    .pipe(process.stdout);
};

jmdobry avatar Aug 02 '17 22:08 jmdobry