callback not firing when custom storage engine finishes handling file.

[AndrewProbityWeb]

I have started writing a custom storage engine for a cloud storage provider. The file upload works, but when the upload is finished and the cb is invoked, express.js does not move to the next step on the route, causing the request to that route to hang indefinitely. There are no errors on the server and I can't see anything using debug, but the console.log before the cb() works, the console.log in the routes file doesn't.

This is the code for the CustomStorageEngine. The uploadFile function is just a couple of fetch calls to the cloud provider and some error handling. I am new to node/express, is the callback being called in a promise an issue? If I pass an error to the callback it behaves normally.

class CustomStorageEngine implements multer.StorageEngine {
  private nameFn: nameFnType;
  private siteURL: string;

  constructor(opts: Options) {
    this.nameFn = opts.nameFn || defaultNameFn;
    this.siteURL = opts.siteUrl
  }

  _handleFile = (
    req: Request,
    file: Express.Multer.File,
    cb: (error?: any, info?: CustomFileResult) => void
  ): void => {
    
    const fileName = this.nameFn(req, file);

    uploadFile(fileName, file, this.siteURL)
      .then((asset: CloudAsset) => {
        console.log(asset); // this fires

        cb(null, { id: asset.id, name: asset.name, url: asset.url })
      })
      .catch((error) => {
        cb(error)
      })
  };

  _removeFile = (
    _req: Request,
    file: Express.Multer.File & { name: string },
    cb: (error: Error | null) => void
  ): void => {
    cb(null)
  };
}

export default (opts: Options) => {
  return new CustomStorageEngine(opts);
};

This is the code for the route.

const router = Router();

const uploader = multer({
storage: StorageEngine(
    {
      siteUrl: config.siteUrl
    }
  )
})

router.post('/asset', uploader.single('file'), async (req, res) => {
  console.log(req.file) // this never fires
  res.status(200).json({
    "id": (req.file as CustomFileResult).id,
    "url": (req.file as CustomFileResult).url,
    "name": (req.file as CustomFileResult).name,
    "physical_file_path": (req.file as CustomFileResult).physicalFilePath
  });
});


export default router;

Any help would be greatly appreciated

[tonypeng]

were you able to resolve this issue? running into something similar

[EinfachHans]

Any solution here? I tried to setup a very simple engine that nearly does nothing and just calls the callback function, which doesn't work. This part is never called and the next function is never called:

busboy.on('close', function () {
  readFinished = true
  indicateDone()
})

[dan-garden]

Running to this issue too. Could it be an issue with a newer version of multer?

[heennkkee]

Without digging to deep into how multer does on the inside, it seems like you need to make sure you consume the stream from the file, otherwise the middleware won't continue on.

Something like this works for me as a POC

import multer from 'multer';
import stream from "node:stream";
...
const storage: multer.StorageEngine = {
  _handleFile: function _handleFile(req, file, cb) {

     // Create a writeable stream to write to
      const dummyStream = new stream.Writable();
      dummyStream._write = function (_chunk, _encoding, done) {
        console.log('Writing to stream');
        done();
      };

      dummyStream.on('finish', function () {
        cb(null, {});
      });

      // Pipe the files' stream to the dummystream so we consume it
      file.stream.pipe(dummyStream);
    },
    _removeFile: function _removeFile(req, file, cb) {}
};