kafka-connect-file-pulse
kafka-connect-file-pulse copied to clipboard
feat(plugin): Delete the compressed file after extraction
feat: (plugin): Add a configuration to delete the compressed file after extraction
Add the flag fs.delete.compress.files.enabled
indicating whether compressed file should be deleted after extraction.
This flag is added to avoid an infinit loop when using a compressed file with the following configuration:
"offset.attributes.string": "name+lastModified",
"fs.cleanup.policy.class": "io.streamthoughts.kafka.connect.filepulse.fs.clean.LocalMoveCleanupPolicy",
Im fact, if the compressed file is not removed, once extracted, the extracted file is processed, and move to the preconfigured folder. Therefore, the compressed file will be extrated again with a new lastModified value, and the extracted file would be processed again.
When using zipped file, is there any reason , why the zipped file is not removed from the directoty once unzipped? #603