tajo icon indicating copy to clipboard operation
tajo copied to clipboard

Published 20 hours ago • verified publisher icon apache

TAJO-1480: Kafka Consumer for kafka strage.

Open hys9958 opened this issue 9 years ago • 3 comments

Hi guys, I have implemented for this issue and unit tests. Please review my patch.

Thanks.

hys9958 avatar Mar 30 '15 18:03 hys9958

Oops! My mistake. Hi, @blrunner I have modified it.(0.11.0-SNAPSHOT)

hys9958 avatar Apr 05 '15 13:04 hys9958

I found many dependencies as follows.

[INFO] ------------------------------------------------------------------------
[INFO] Building Tajo Kafka Storage 0.11.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom (3 KB at 2.2 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.pom (2 KB at 5.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloading: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.pom (2 KB at 6.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloading: https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom
Downloaded: https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.1.6/snappy-java-1.1.1.6.pom (4 KB at 10.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloading: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom
Downloaded: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.pom (2 KB at 6.3 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloading: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom
Downloaded: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.pom (11 KB at 33.0 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloading: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.3/zkclient-0.3.pom
Downloaded: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.3/zkclient-0.3.pom (3 KB at 9.7 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.3.1/zookeeper-3.3.1.pom (3 KB at 6.8 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/testing/0.88/testing-0.88.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/testing/0.88/testing-0.88.pom
Downloading: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.pom
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.pom (2 KB at 4.1 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/airlift/0.88/airlift-0.88.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/airlift/0.88/airlift-0.88.pom
Downloading: https://repo.maven.apache.org/maven2/io/airlift/airlift/0.88/airlift-0.88.pom
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/airlift/0.88/airlift-0.88.pom (10 KB at 21.3 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/airbase/20/airbase-20.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/airbase/20/airbase-20.pom
Downloading: https://repo.maven.apache.org/maven2/io/airlift/airbase/20/airbase-20.pom
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/airbase/20/airbase-20.pom (59 KB at 61.5 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloading: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom
Downloaded: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.pom (764 B at 2.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.pom (9 KB at 26.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-parent/0.5/bval-parent-0.5.pom (25 KB at 52.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.pom (6 KB at 19.4 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloading: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.pom
Downloaded: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.pom (3 KB at 9.7 KB/sec)
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/io/airlift/testing/0.88/testing-0.88.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloading: https://repository.jboss.org/nexus/content/repositories/releases/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/io/airlift/testing/0.88/testing-0.88.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloading: http://repo2.maven.org/maven2/org/eclipse/jetty/jetty-distribution/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar
Downloading: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar
Downloading: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar
Downloading: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar
Downloaded: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/3.2/jopt-simple-3.2.jar (52 KB at 57.5 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar
Downloaded: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar (25 KB at 81.4 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.jar
Downloaded: https://repo.maven.apache.org/maven2/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar (162 KB at 109.2 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar
Downloaded: https://repo.maven.apache.org/maven2/io/airlift/testing/0.88/testing-0.88.jar (26 KB at 47.8 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka-clients/0.8.2.0/kafka-clients-0.8.2.0.jar (317 KB at 163.2 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.jar
Downloaded: https://repo.maven.apache.org/maven2/com/google/code/findbugs/annotations/2.0.2/annotations-2.0.2.jar (73 KB at 129.0 KB/sec)
Downloading: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.jar
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-core/0.5/bval-core-0.5.jar (61 KB at 166.2 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/com/101tec/zkclient/0.4/zkclient-0.4.jar (64 KB at 120.3 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/apache/bval/bval-jsr303/0.5/bval-jsr303-0.5.jar (242 KB at 236.5 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.jar (3900 KB at 372.1 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar (6960 KB at 537.3 KB/sec)

I would like it more simple if you exclude unnecessary dependencies.

blrunner avatar Apr 06 '15 01:04 blrunner

@blrunner, Thanks for your nice comment. I 'll fix it and update soon.

hys9958 avatar Apr 06 '15 01:04 hys9958