elasticsearch-jdbc
elasticsearch-jdbc copied to clipboard
Example for mysql not work for me
This is example file.sh for sync data from mysql to elasticsearch.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
bin=${DIR}/../bin
lib=${DIR}/../lib
echo '
{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/crmtest",
"user" : "test123",
"schedule" : "0 0-59 0-23 ? * *",
"password" : "test1234@123A",
"sql" : [{"statement": "select id as _id, full_name as full_name from user"}],
"index": "crmtest",
"type": "user",
"metrics": {
"enabled": true
},
"elasticsearch" : {
"cluster" : "elasticsearch",
"host" : "localhost",
"port" : 9200
}
}
}
' | java \
-cp "${lib}/*" \
-Dlog4j.configurationFile=${bin}/log4j2.xml \
org.xbib.tools.Runner \
org.xbib.tools.JDBCImporter
I already have
commons-cli-1.3.1.jar jdbc-driver-csv-1.0.0.jar lucene-misc-5.5.0.jar
compress-lzf-1.0.2.jar joda-convert-1.2.jar lucene-queries-5.5.0.jar
derby-10.12.1.1.jar joda-time-2.9.4.jar lucene-queryparser-5.5.0.jar
elasticsearch-2.3.4.jar jsr166e-1.1.0.jar lucene-sandbox-5.5.0.jar
elasticsearch-helper-2.3.4.0.jar jts-1.13.jar lucene-spatial3d-5.5.0.jar
elasticsearch-jdbc-2.3.4.0.jar log4j-api-2.5.jar lucene-spatial-5.5.0.jar
guava-18.0.jar log4j-core-2.5.jar lucene-suggest-5.5.0.jar
h2-1.4.191.jar log4j-slf4j-impl-2.5.jar mysql-connector-java-5.1.38.jar
HdrHistogram-2.1.6.jar lucene-analyzers-common-5.5.0.jar netty-3.10.5.Final.jar
hppc-0.7.1.jar lucene-backward-codecs-5.5.0.jar postgresql-9.4.1208.jre7.jar
hsqldb-2.3.3.jar lucene-core-5.5.0.jar securesm-1.0.jar
jackson-core-2.6.6.jar lucene-grouping-5.5.0.jar slf4j-api-1.7.12.jar
jackson-dataformat-cbor-2.6.6.jar lucene-highlighter-5.5.0.jar snakeyaml-1.15.jar
jackson-dataformat-smile-2.6.6.jar lucene-join-5.5.0.jar spatial4j-0.5.jar
jackson-dataformat-yaml-2.6.6.jar lucene-memory-5.5.0.jar t-digest-3.0.jar
mysql-connector-java on lib folder but when I run this script. It couldn't select anything.
✘ ⚙ begood@begood ~/work/elasticsearch-jdbc-2.3.4.0/bin ./mysql-test.sh
./mysql-test.sh: 19: ./mysql-test.sh: Bad substitution
[21:41:30,021][INFO ][metrics.source.plain ][pool-5-thread-1] totalrows = 0, 29 seconds = 29989 ms, 0 = 0.0 bytes bytes, 0.0 bytes = 0 avg size, 0 dps, 0 MB/s
[21:41:30,021][INFO ][metrics.sink.plain ][pool-5-thread-1] 29 seconds = 29499 ms, submitted = 0, succeeded = 0, failed = 0, 0 = 0.0 bytes bytes, 0.0 bytes = 0 avg size, 0 dps, 0 MB/s
[21:42:00,019][INFO ][metrics.source.plain ][pool-5-thread-1] totalrows = 0, 59 seconds = 59989 ms, 0 = 0.0 bytes bytes, 0.0 bytes = 0 avg size, 0 dps, 0 MB/s
Please help me fix this! Note: I have data in database
The elasticsearch port should be 9300 as it uses TransportClient