DataX icon indicating copy to clipboard operation
DataX copied to clipboard

新增datax对Sybase数据库的支持

Open cj1234567 opened this issue 6 years ago • 13 comments

已开发测试datax对Sybase数据库的读写支持

cj1234567 avatar Feb 06 '19 11:02 cj1234567

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Feb 06 '19 11:02 CLAassistant

I'd like to install the sybase reader, where I can download?

alexcool2016 avatar Mar 07 '19 06:03 alexcool2016

Got exception below when trying to use sybasereader:

2019-03-07 15:07:52.248 [job-0] ERROR JobContainer - Exception when job run java.lang.NoSuchFieldError: Sybase at com.alibaba.datax.plugin.reader.sybasereader.SybaseReader.(SybaseReader.java:17) ~[sybasereader-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.reader.sybasereader.SybaseReader$Job.init(SybaseReader.java:33) ~[sybasereader-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673) ~[datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303) ~[datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) ~[datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.start(Engine.java:92) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.entry(Engine.java:171) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.main(Engine.java:204) [datax-core-0.0.1-SNAPSHOT.jar:na] 2019-03-07 15:07:52.253 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00% 2019-03-07 15:07:52.255 [job-0] ERROR Engine -

alexcool2016 avatar Mar 07 '19 07:03 alexcool2016

I'd like to install the sybase reader, where I can download?

Have you downloaded the code?

cj1234567 avatar Mar 07 '19 07:03 cj1234567

I'd like to install the sybase reader, where I can download?

Here is the link,https://github.com/cj1234567/DataX

cj1234567 avatar Mar 07 '19 07:03 cj1234567

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

alexcool2016 avatar Mar 07 '19 08:03 alexcool2016

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

Yes,some core code has been added. And because the whole Compiled datax jar package is too large,almost 900M,i will push it to my baidu network disk and then share it to you

cj1234567 avatar Mar 07 '19 08:03 cj1234567

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

Yes,some core code has been added. And because the whole Compiled datax jar package is too large,almost 900M,i will push it to my baidu network disk and then share it to you

Thanks! But you can just release the ones relate to sybasereader, then the sizing will be small

alexcool2016 avatar Mar 07 '19 08:03 alexcool2016

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

Yes,some core code has been added. And because the whole Compiled datax jar package is too large,almost 900M,i will push it to my baidu network disk and then share it to you

Thanks! But you can just release the ones relate to sybasereader, then the sizing will be small

Here is the link,https://pan.baidu.com/s/1hjUGuSKDM4tqeaytzK5R2A

cj1234567 avatar Mar 07 '19 09:03 cj1234567

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

Yes,some core code has been added. And because the whole Compiled datax jar package is too large,almost 900M,i will push it to my baidu network disk and then share it to you

Thanks! But you can just release the ones relate to sybasereader, then the sizing will be small

Here is the link,https://pan.baidu.com/s/1hjUGuSKDM4tqeaytzK5R2A

Downloading, many thanks!

alexcool2016 avatar Mar 07 '19 09:03 alexcool2016

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

Yes,some core code has been added. And because the whole Compiled datax jar package is too large,almost 900M,i will push it to my baidu network disk and then share it to you

Thanks! But you can just release the ones relate to sybasereader, then the sizing will be small

Here is the link,https://pan.baidu.com/s/1hjUGuSKDM4tqeaytzK5R2A

Downloading, many thanks!

Hope it would help you

cj1234567 avatar Mar 07 '19 09:03 cj1234567

I just download the sybasereader folder files(jar and json), any else? Or do i have to download all? Do you have the whole release version (already complied and packaged) URL?

Yes,some core code has been added. And because the whole Compiled datax jar package is too large,almost 900M,i will push it to my baidu network disk and then share it to you

Thanks! But you can just release the ones relate to sybasereader, then the sizing will be small

Here is the link,https://pan.baidu.com/s/1hjUGuSKDM4tqeaytzK5R2A

Downloading, many thanks!

Hope it would help you

It works! Thank you very much @cj1234567 !

alexcool2016 avatar Mar 08 '19 06:03 alexcool2016

@cj1234567 你好,我的配置如下,报错,源和目标字段类型对应的,不知道为何说是 “Implicit conversion from datatype 'VARCHAR' to 'INT' is not allowed“ { "job": { "content": [{ "reader": { "name": "mysqlreader", "parameter": { "column": ["id","user_id", "vote_id","group_id","create_time"], "connection": [{ "jdbcUrl": ["jdbc:mysql://192.168.0.210:3306/dataxTest"], "table": ["vote_record"] }], "password": "123456", "username": "root" } }, "writer": {
"name": "sybasewriter", "parameter": { "column": ["id","user_id", "vote_id","group_id","create_time"], "connection": [{ "jdbcUrl": "jdbc:sybase:Tds:192.168.0.183:8000/tempdb", "table": ["tempdb.dbo.vote_record1"] }], "password": "myPassword", "username": "sa" } } }], "setting": { "speed": { "channel": 10 } } } }

mysql 建表语句: CREATE TABLE vote_record ( id int(11) NOT NULL AUTO_INCREMENT, user_id varchar(20) NOT NULL DEFAULT '' COMMENT '用户Id', vote_id int(10) unsigned NOT NULL DEFAULT '0' COMMENT '投票数', group_id int(10) unsigned NOT NULL DEFAULT '0' COMMENT '用户组id 0-未激活用户 1-普通用户 2-vip用户 3-管理员用户', create_time datetime NOT NULL DEFAULT '1971-01-01 01:01:01' COMMENT '创建时间', PRIMARY KEY (id) ) ENGINE=InnoDB AUTO_INCREMENT=20000001 DEFAULT CHARSET=utf8 COMMENT='投票记录表';

sybase建表语句如下: CREATE TABLE tempdb.dbo.vote_record1 ( id int NOT NULL, user_id varchar(20) NOT NULL, vote_id int NOT NULL, group_id int NOT NULL, create_time datetime NOT NULL ) GO

报错如下: 2019-12-14 16:17:17.119 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: {"exception":"Implicit conversion from datatype 'VARCHAR' to 'INT' is not allowed. Use the CONVERT function to run this query.\n","record":[{"byteSize":1,"index":0,"rawData":9,"type":"LONG"},{"byteSize":1,"index":1,"rawData":"9","type":"STRING"},{"byteSize":1,"index":2,"rawData":9,"type":"LONG"},{"byteSize":1,"index":3,"rawData":9,"type":"LONG"},{"byteSize":8,"index":4,"rawData":1575420223000,"type":"DATE"}],"type":"writer"} 2019-12-14 16:17:20.376 [0-0-0-writer] WARN CommonRdbmsWriter$Task - 回滚此次写入, 采用每次写入一行方式提交. 因为:JZ0BE: BatchUpdateException:执行批处理语句时出错:Implicit conversion from datatype 'VARCHAR' to 'INT' is not allowed. Use the CONVERT function to run this query.

lingzhy avatar Dec 14 '19 08:12 lingzhy