streamx
streamx copied to clipboard
qubole
kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)
I noticed I have to configure the hadoop config files like core-site.xml, hdfs-site.xml to configure S3. And I could not find the mentioned config/hadoop-conf in my installation (Kafka 0.10.2.0). So...
I see support added for IAM Roles, but, readme never explains that.
**I am trying to export the avro records from kafka and load them into s3 in parquet format. I got an exception as shown below.** ERROR WorkerSinkTask{id=s3-sink-0} Task threw an...
Hi I am getting below issue while submitting job. Could you please help me on this issue? {"error_code":400,"message":"Connector configuration is invalid and contains the following 5 error(s):\nNot a valid partitioner...
I have a question, data in kafka is in json format, in each event I have a field called"eventTimestamp" which is a long number which represents the event time ,...
This allows the connector to work with Kafka 0.11. I also upgraded Guava to 20.0 to fix the `NoSuchMethodError` I was seeing under #48.
I use streamx to sink kafka data to S3 as parquet files, everything is fine, I can observe the logs, which give me the messages, that the parquet fiiles are...
I want the message published in kafka topics to be stored in s3 bucket as text/json. Can I use StreamX as alternative to Aws Kinesis Firehose delivery stream.The Delivery stream...
Hi I'd like to know if there is an option to write one file per partition which means per hour. For example, if i have 5 workers with 5 tasks,...
