datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Version 0.6.0 Panic error during SQL execution

Open ike560 opened this issue 3 years ago • 3 comments

Describe the bug Panic error occurs when executing Join SQL. Refer to the attached jupyter execution result.

Exception: Arrow error: External error: Arrow error: External error: Execution error: Join Error: task 57 panicked thread 'tokio-runtime-worker' panicked at 'range end index 83 out of range for slice of length 72'.

datafusion_panicError

Additional context In version 0.5.2, No problem with the same SQL.

ike560 avatar Jun 16 '22 09:06 ike560

Additional Information

Column (company) value is UTF-8 Japanese string. In company value is alpha-numeric string, there is no problem with the same SQL.

ghost avatar Jul 04 '22 04:07 ghost

Solved!

After modifying the SQL as below, it was OK. Help. Do you know why this modified SQL is OK? Before  GROUP BY company After   GROUP BY company, stock_code

sql = '''
SELECT mas.company, SUM(CAST(trading_volume AS INTEGER)/10000000) AS sum_trading_volume FROM ( 
  SELECT stock_code, company FROM stock_master) AS mas 
    INNER JOIN stock_trading_info 
    USING (stock_code) 
  GROUP BY company, stock_code
  ORDER BY sum_trading_volume DESC 
  LIMIT 10
  '''
df = ctx.sql(sql)

ghost avatar Jul 04 '22 05:07 ghost

@ike560 it is valid sql. You can group by company, stock_code but still have only company in select. That doesn't make much sense, but it is still valid

comphead avatar Jul 08 '22 04:07 comphead