kafka-connect-file-pulse
kafka-connect-file-pulse copied to clipboard
streamthoughts
🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
File-Pulse: 2.7.0 Kafka: 3.2.1 Confluent-Platform: 7.2.1 We are creating a connector with the following configuration: ``` { "name": "file-pulse-connector", "config": { "connector.class": "io.streamthoughts.kafka.connect.filepulse.source.FilePulseSourceConnector", "fs.cleanup.policy.class": "io.streamthoughts.kafka.connect.filepulse.fs.clean.LogCleanupPolicy", "fs.listing.class": "io.streamthoughts.kafka.connect.filepulse.fs.LocalFSDirectoryListing", "fs.listing.directory.path": "/data/", "fs.listing.interval.ms":...
Bumps `org.apache.kafka.version` from 3.2.0 to 3.2.1. Updates `kafka-clients` from 3.2.0 to 3.2.1 Updates `connect-api` from 3.2.0 to 3.2.1 Dependabot will resolve any conflicts with this PR as long as you...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps [azure-identity](https://github.com/Azure/azure-sdk-for-java) from 1.2.5 to 1.5.4. Release notes Sourced from azure-identity's releases. azure-identity_1.5.4 1.5.4 (2022-08-08) Bugs Fixed Fixes IntelliJCredential 21150 Fixes AzureCliCredential to properly respect tenant IDs. Other Changes Dependency...
Bumps [google-cloud-nio](https://github.com/googleapis/java-storage-nio) from 0.122.11 to 0.124.14. Release notes Sourced from google-cloud-nio's releases. v0.124.14 0.124.14 (2022-08-08) Dependencies update dependency com.google.cloud:google-cloud-storage to v2.11.3 (#977) (f044437) v0.124.13 0.124.13 (2022-08-05) Dependencies update dependency com.google.cloud:google-cloud-storage...
Bumps nexus-staging-maven-plugin from 1.6.8 to 1.6.13. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a...
Bumps `aws-java-sdk.version` from 1.12.210 to 1.12.275. Updates `aws-java-sdk-s3` from 1.12.210 to 1.12.275 Changelog Sourced from aws-java-sdk-s3's changelog. 1.12.275 2022-08-03 AWS WAFV2 Features You can now associate an AWS WAF web...
Hello, what I want to ask is about the usage of the columns key under DelimitedRowFilter in Processing Filters. There is no example of this parameter in your document.
Bumps [aws-java-sdk-s3](https://github.com/aws/aws-sdk-java) from 1.12.210 to 1.12.261. Changelog Sourced from aws-java-sdk-s3's changelog. 1.12.261 2022-07-14 AWS Config Features Update ResourceType enum with values for Route53Resolver, Batch, DMS, Workspaces, Stepfunctions, SageMaker, ElasticLoadBalancingV2, MSK...
Team, Is there a roadmap for additional file readers like a Parquet file reader? Thanks.
**Context** I'm using FilePulse to pick up JSON files from a GCS bucket being published by a proprietary vendor. These files are published as arrays of JSON documents (around 15MB...
Metadata
Owner
Metadata
🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka