embulk-input-jdbc icon indicating copy to clipboard operation
embulk-input-jdbc copied to clipboard

Embulk freezes when I specify where condition

Open OmkarPathak opened this issue 5 years ago • 4 comments

Hi team,

I am using embulk-input-oracle for fetching data. This works fine when I write query without any WHERE clause. But when I specify a WHERE clause it just freezes.

This is the config we are using:

in:
  type: oracle
  driver_path: ojdbc7-12.1.0.2.jar
  url: jdbc:oracle:thin:@something.com:1526/DB
  user: ABC
  password: "PASSWORD"
  query: "SELECT * FROM SCHEMA.TABLE WHERE ABC_DATE > to_date('01 JAN 2019','DD MON YY')"
  fetch_rows: 20000
  connect_timeout: 100

Is there any flag to be set for this to work?

OmkarPathak avatar Mar 06 '20 05:03 OmkarPathak

Hello, @OmkarPathak

  • How many records do you use?
  • Have you ever tried the same SQL with PL/SQL? (without embulk)

I suspect that SQL doesn't use the index properly.

You can run more verbosely using embulk run -l debug config.yml

hiroyuki-sato avatar Mar 06 '20 05:03 hiroyuki-sato

@hiroyuki-sato yes the same query works well in oracle sql. Also tried running verbosely, but it still freezes and no output is generated at sysout (also no error is displayed, and program doesn't exit either)

OmkarPathak avatar Mar 06 '20 05:03 OmkarPathak

Hello, @OmkarPathak

  • How about embulk preview?
  • How about increase memory like embulk -J-Xmx10g run config.yml (-J option must set before run)

hiroyuki-sato avatar Mar 06 '20 05:03 hiroyuki-sato

Hi @OmkarPathak Would you show the log until embulk freezes?

hito4t avatar Mar 11 '20 05:03 hito4t