rclone-js icon indicating copy to clipboard operation
rclone-js copied to clipboard

How to stream from google drive (googleapis), instead of using fetch?

Open gemini0x2 opened this issue 1 year ago • 0 comments

First of all, thanks for making this project. After doing some tests I got my video files to stream on the browser, except for some large files (~30gbs), which did not stream even after increasing the chunkSize. The video played only after all segments (~30gbs) were downloaded by the browser. If you know an optimal chunkSize for large video files do let me know please.

Anyways, I made this issue because I'm trying to stream a file from google drive, instead of using fetch() I want to use my function getFileContent() to get a file from google drive, but the file contents can only be returned as a node PassThrough stream or an ArrayBuffer. On the other hand, I noticed that fetch() returns a ReadableStream in the response.body. Hence, I tried converting the node PassThrough stream into a web streams api, but all I get is Error: Magic is wrong.

Below is what I currently have.

export async function getFileContent(fileId, head) {
  const response = await drive.files.get(
    {
      fileId,
      alt: 'media',
      header: head,
      method: 'GET',
      mimeType: 'application/octet-stream',
    }, {
      responseType: 'stream' // returns a PassThrough stream
      // responseType: 'arraybuffer' // returns an ArrayBuffer with [Uint8Contents]
    });

   response.data.on('error', err => {
     console.log(`Error reading file ${fileId}:`, err);
     response.data.sendStatus(500);
   });

   response.data.on('end', () => {
     console.log(`File ${fileId} downloaded successfully.`);
     response.data.sendStatus(200);
   });

  // // check that the response contains data
  // response.data.on('data', (chunk) => {
  //   console.log(`Received ${chunk.length} bytes of data`);
  // });

   // Trying to convert node stream to web stream
  const nodeStream = response.data;
  const webStream = await new ReadableStream({
    start(controller) {
      nodeStream.on('data', chunk => {
        controller.enqueue(chunk);
      });

      nodeStream.on('end', () => {
        controller.close();
      });

      nodeStream.on('error', err => {
        controller.error(err);
      });
    }
  });

  return webStream;
}

gemini0x2 avatar May 04 '23 17:05 gemini0x2