chdb icon indicating copy to clipboard operation
chdb copied to clipboard

Code: 53. DB::Exception: Unrecognized data type: datetime64[ms, UTC] on column ts. (TYPE_MISMATCH)

Open jovezhong opened this issue 7 months ago • 2 comments

Please make sure that the version you're using is still supported (you can find the list here).

You have to provide the following information whenever possible.

Describe what's wrong

A clear and concise description of what works not as it is supposed to.

A link to reproducer in https://fiddle.clickhouse.com/.

Does it reproduce on the most recent release?

The list of releases

Enable crash reporting

Change "enabled" to true in "send_crash_reports" section in config.xml:

<send_crash_reports>
        <!-- Changing <enabled> to true allows sending crash reports to -->
        <!-- the ClickHouse core developers team via Sentry https://sentry.io -->
        <enabled>false</enabled>

How to reproduce

  • Which ClickHouse server version to use Latest chdb 3.1.2
  • Which interface to use, if it matters
  • Non-default settings, if any
  • CREATE TABLE statements for all tables involved
  • Sample data for all these tables, use clickhouse-obfuscator if necessary
  • Queries to run that lead to an unexpected result
items=chdb.query("select parseDateTimeBestEffort(ts_str,3,'UTC') as ts, * from file('costco_details.csv','CSV','ts_str String, total_amount Decimal32(2), warehouse String,item_name String,item_code String,item_amount Decimal32(2),item_note String,subtotal Decimal32(2),tax Decimal32(2),payment_method String, total_items_sold Int8, savings Decimal32(2)') settings input_format_csv_skip_first_lines = 1",output_format="DataFrame")
chdb.query("select * from Python(items)",output_format="DataFrame")
Image

the first dataframe can be shown in marimo, but select * failed with DBerror (should not related to marimo) Expected behavior

A clear and concise description of what you expected to happen.

Error message and/or stacktrace

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

jovezhong avatar Apr 16 '25 01:04 jovezhong

I guess when chdb exports the SQL results as a dataframe, if the column in in datetime64(..), read it again with Python(df) will lose some info. I got the same error select toStartOfDay(ts) from Python(items)

One workaround is not to include datetime in the 1st query and use string to share datetime values, which is a bit ugly

jovezhong avatar Apr 16 '25 01:04 jovezhong

Already fixed after upgrading new ch core version(https://github.com/chdb-io/chdb/pull/383)

kafka1991 avatar Oct 11 '25 02:10 kafka1991