fix: shutdown of connection to sink before publishing last record
Describe the bug
The last stream of records is not published in the sink if we use the Close() function in the sink to close the connection to the sink. In agent.go the sinks are closed when the stream closes, but the stream closes once all records are emitted from extractor, hence while publishing the last record we get an error that the connection has been closed. This error has not caused any issue with sinks yet, because most sinks do not use the Close() in the sink for anything.
To Reproduce Steps to reproduce the behavior:
- Go to
file sink - In close function, uncomment the
return s.File.Close() - Run a
recipewith sink as File Sink.
Expected behavior
With proper config in the recipe, all records must be published in the sink before Close() is called.
Screenshots
Additional context
The file sink was allowed for now, because not closing the File was fine and golang's garbage collector anyway closes it after the job ends. But it will be great to rethink how we close the stream and sinks altogether in the agent to make Close() for sink useful.