canal
canal copied to clipboard
canal-adapter同步mysql至es,etlCondition无效
Question
我的conf/es/test.yml内容如下:
dataSourceKey: defaultDS destination: example1 groupId: g1 esMapping: _index: test6-subaccount_history _type: _doc _id: _id upsert: true #pk: id sql: "select f_id as _id, f_id, f_version, f_user_id, f_account_id, f_subaccount_id, f_account_type, f_subaccount_type, f_action_type, f_currency, f_amount, f_balance, f_suspense, f_clearing, f_total, f_action_id, f_source_id, f_source_type, f_source_type_code, f_source_ext, f_created_at, f_updated_at from t_subaccount_history" #objFields:
#_labels: array:; etlCondition: "where f_version>3" commitBatch: 3000
不加etl条件可以正常的同步至es,但加上etl条件,依旧是把所有mysql更新或新增记录都同步到es,仿佛etl条件无效
我也遇到了同样的问题
Question
我的conf/es/test.yml内容如下:
dataSourceKey: defaultDS destination: example1 groupId: g1 esMapping: _index: test6-subaccount_history _type: _doc _id: _id upsert: true #pk: id sql: "select f_id as _id, f_id, f_version, f_user_id, f_account_id, f_subaccount_id, f_account_type, f_subaccount_type, f_action_type, f_currency, f_amount, f_balance, f_suspense, f_clearing, f_total, f_action_id, f_source_id, f_source_type, f_source_type_code, f_source_ext, f_created_at, f_updated_at from t_subaccount_history" #objFields: #_labels: array:; etlCondition: "where f_version>3" commitBatch: 3000
不加etl条件可以正常的同步至es,但加上etl条件,依旧是把所有mysql更新或新增记录都同步到es,仿佛etl条件无效
请问你这个问题解决了吗?
etlCondition:条件无效
同问
我也遇到了,1.1.5版本
可能是yaml格式的问题,etlCondition字段需要在 esMapping子级
也是醉了,这文档写的跟啥一样 etlCondition不支持直接写死的方式,如下 etlCondition: "where f_version>3"
只能这样etlCondition: "where f_version>{}" 然后再调用etl全量同步接口的时候 加上参数 ?params=3 这样才会生效
也是醉了,这文档写的跟啥一样 etlCondition不支持直接写死的方式,如下 etlCondition: "where f_version>3"
只能这样etlCondition: "where f_version>{}" 然后再调用etl全量同步接口的时候 加上参数 ?params=3 这样才会生效
请问如果有多个条件呢?etlcondition 怎么写?还有etl指令怎么拼接参数?
也是醉了,这文档写的跟啥一样 etlCondition不支持直接写死的方式,如下 etlCondition: "where f_version>3" 只能这样etlCondition: "where f_version>{}" 然后再调用etl全量同步接口的时候 加上参数 ?params=3 这样才会生效
请问如果有多个条件呢?etlcondition 怎么写?还有etl指令怎么拼接参数?
用分号分隔 etlCondition: "where id>={} and age = {}" curl http://127.0.0.1:8081/etl/rdb/mysql1/mytest_user.yml?params=4;44 -X POST
那如果是增量呢?怎么处理过滤条件?