dask-contrib
[BUG][GPU Logic Bug] "SELECT (<string>)||(<column(decimal)>) FROM <table>" brings Error
What happened:
"SELECT (<string>)||(<column(decimal)>) FROM <table>" brings different results, when using CPU and GPU.
What you expected to happen:
It is the same result, when using CPU and GPU.
Minimal Complete Verifiable Example:
import pandas as pd
import dask.dataframe as dd
from dask_sql import Context
c = Context()
df = pd.DataFrame({
'c0': [0.5113391810437729]
})
t1 = dd.from_pandas(df, npartitions=1)
c.create_table('t1', t1, gpu=False)
c.create_table('t1_gpu', t1, gpu=True)
print('CPU Result:')
result1= c.sql("SELECT ('A')||(t1.c0) FROM t1").compute()
print(result1)
print('GPU Result:')
result2= c.sql("SELECT ('A')||(t1_gpu.c0) FROM t1_gpu").compute()
print(result2)
Result:
CPU Result:
Utf8("A") || t1.c0
0 A0.5113391810437729
GPU Result:
Utf8("A") || t1_gpu.c0
0 A0.511339181
Anything else we need to know?:
Environment:
- dask-sql version: 2023.6.0
- Python version: Python 3.10.11
- Operating System: Ubuntu22.04
- Install method (conda, pip, source): Docker deploy by https://hub.docker.com/layers/rapidsai/rapidsai-dev/23.06-cuda11.8-devel-ubuntu22.04-py3.10/images/sha256-cfbb61fdf7227b090a435a2e758114f3f1c31872ed8dbd96e5e564bb5fd184a7?context=explore
Trying out your reproducer with latest main gives me an error 😕 looks like at some point between now and 2023.6.0 our logical plan has changed such that we skip the casting of the non-string column:
# 2023.6.0
Projection: Utf8("A") || CAST(t1.c0 AS Utf8)
TableScan: t1 projection=[c0]
# main
Projection: Utf8("A") || t1.c0
TableScan: t1 projection=[c0]
Leading to errors in the binary operation; cc @jdye64 if you have any capacity to look into this. As for the original issue, it seems like that generally comes down to difference in the behavior of cast operations on CPU/GPU, as the following shows the same issue:
print('CPU Result:')
result1= c.sql("SELECT CAST(c0 AS STRING) FROM t1").compute()
print(result1)
print('GPU Result:')
result2= c.sql("SELECT CAST(c0 AS STRING) FROM t1_gpu").compute()
print(result2)
Can look into that, would you mind modifying your issue description / title to reflect this?
Thanks for your confirmation. We look forward to your replies about bug fixes.