multer icon indicating copy to clipboard operation
multer copied to clipboard

[FEAT] Google Cloud Functions Support

Open emregunel opened this issue 7 months ago • 3 comments

Add streamHandler option for Google Cloud Functions support

Problem

Multer currently uses req.pipe(busboy) to process multipart form data. This works well in most environments, but causes issues in Google Cloud Functions (GCF) where the request body is pre-processed and made available as req.rawBody. When Multer tries to pipe the request in GCF, it fails because the request stream has already been consumed.

Solution

This PR adds a new streamHandler option that allows customizing how request data is fed to busboy. Instead of hardcoding req.pipe(busboy), the middleware now accepts a custom function that can handle both standard Express environments and pre-processed environments like GCF.

Implementation Details

  1. Added a new streamHandler configuration option to the Multer constructor
  2. Created a default stream handler that maintains the existing req.pipe(busboy) behavior
  3. Updated the middleware to use the custom handler when provided
  4. Modified the cleanup logic to only unpipe when using the default pipe-based handler
  5. Added documentation explaining the new option and its usage
  6. Added comprehensive tests for both default and custom handlers

Example Usage

For Google Cloud Functions or similar environments where the request body is pre-processed:

const multer = require('multer')

// Create a custom stream handler that works with pre-processed bodies
function gcfStreamHandler(req, busboy) {
  if (req.rawBody) {
    // In Google Cloud Functions, use the pre-processed body
    busboy.end(req.rawBody)
  } else {
    // Fall back to standard behavior in other environments
    req.pipe(busboy)
  }
}

// Create multer instance with custom stream handler
const upload = multer({ 
  storage: multer.memoryStorage(),
  streamHandler: gcfStreamHandler
})

// Use the middleware as usual
app.post('/upload', upload.single('file'), (req, res) => {
  res.send('File uploaded successfully')
})

Benefits

  1. Compatibility: Makes Multer work seamlessly in Google Cloud Functions and similar environments
  2. Flexibility: Provides a customization point for how request data is fed to busboy
  3. Backwards compatibility: Default behavior remains unchanged
  4. Minimal changes: Small, targeted modification to the codebase

Testing

Added new tests that verify:

  1. The default behavior continues to work as expected
  2. The custom stream handler works properly with simulated GCF-style pre-processed bodies

All existing tests continue to pass.

Documentation

Added the new streamHandler option to the README.md with example usage for Google Cloud Functions environments.

Related Issues

This PR resolves the issue described in https://github.com/expressjs/multer/issues/1189 where Multer breaks in Google Cloud Functions due to pre-processed request bodies.

emregunel avatar May 11 '25 11:05 emregunel

CI initialization errors will be solved once https://github.com/expressjs/multer/pull/1308 is merged :+1:

UlisesGascon avatar May 11 '25 17:05 UlisesGascon

Thanks @emregunel for this PR! I was checking the CI reports and seems like there are some linting issues, can you run npm run lint -- --fix and commit the changes? Thanks :)

UlisesGascon avatar May 12 '25 06:05 UlisesGascon

Thanks @UlisesGascon for looking at it :) I fixed linting with latest commit

emregunel avatar May 12 '25 10:05 emregunel