s3-upload-stream icon indicating copy to clipboard operation
s3-upload-stream copied to clipboard

parts not fired until the stream is closed

Open saburo70 opened this issue 5 years ago • 0 comments

Hello, I am piping data from a sql request to a csv-stringify then to a upload srteam created with this component. Even tho I set the max concurrent parts to 1, it seems they are flushed only when I close the input stream. I described the issue in stackoverflow here https://stackoverflow.com/questions/55246016/from-aws-rds-sql-server-to-s3-bucket-via-lambda-nodejs-streaming with logging data etc.

The issue with this is that, since I am working in aws lambda, I cannot buffer the entire output (memory resources are usually limited).

Maybe I did something wrong, but from the logging I got it seems that all the multi parts are fired (logging from the part event) AFTER the end method is called.

saburo70 avatar Mar 20 '19 08:03 saburo70