s3-streamlogger icon indicating copy to clipboard operation
s3-streamlogger copied to clipboard

flushFile does not seem to work as intended

Open alexrkopp opened this issue 5 years ago • 3 comments

flushFile calls _upload with the forceNewFile parameter set to true. I'm noticing that when this happens, an empty file is always pushed to S3. Using s3logger._upload(false) seems to be a workaround

alexrkopp avatar Feb 11 '20 15:02 alexrkopp

forceNewFile shouldn't cause an an empty file to be uploaded - the old data should be uploaded to the existing filename, and then new filename used for new data.

_prepareBuffer takes a synchronous copy of the data to be uploaded before the buffer is reset for the new file: https://github.com/Coggle/s3-streamlogger/blob/master/index.js#L169

autopulated avatar Feb 15 '20 15:02 autopulated

(Sorry closed by keyboard shortcut by mistake)

autopulated avatar Feb 15 '20 15:02 autopulated

Actually, the issue might be that we don't save the old filename, so putObject is called with the new filename instead of the old one.

I think here: https://github.com/Coggle/s3-streamlogger/blob/master/index.js#L128 it should use saved.object_name (set here https://github.com/Coggle/s3-streamlogger/blob/master/index.js#L98) instead.

autopulated avatar Feb 15 '20 15:02 autopulated