ibis icon indicating copy to clipboard operation
ibis copied to clipboard

bug: epoch_seconds() method ignores timezone on trino and duckdb

Open vspinu opened this issue 6 months ago • 0 comments

What happened?

epoch_seconds() ignores the time-zone of a timestamp and treats the time-stamp as if a clock-time in UTC:

Trino example:

con = ibis.trino.connect(...)
tbl = con.table("sometable")  # delta table in case that matters


tbl.select(
    var = ibis.literal("2023-01-02")
).mutate(ams=ibis.timestamp(_.var, timezone="Europe/Amsterdam"),
         utc=ibis.timestamp(_.var, timezone="UTC")
).mutate(
    es_ams = ibis.timestamp(_.var, timezone="Europe/Amsterdam").epoch_seconds(),
    es_utc = ibis.timestamp(_.var, timezone="UTC").epoch_seconds(),
    es_ams2 = _.var.cast("Timestamp('Europe/Amsterdam')").epoch_seconds(),
    es_utc2 = _.var.cast("Timestamp('UTC')").epoch_seconds(),
).limit(1).execute()
#           var                       ams                       utc      es_ams      es_utc     es_ams2     es_utc2
# 0  2023-01-02 2023-01-02 01:00:00+01:00 2023-01-02 00:00:00+00:00  1672617600  1672617600  1672617600  1672617600

DuckDB and Polars examples

plcon = ibis.polars.connect()
duckcon = ibis.duckdb.connect()

pltbl = ibis.memtable(pl.DataFrame({"a": 1}))

expr = pltbl.select(
    var = ibis.literal("2023-01-02")
).mutate(ams=ibis.timestamp(_.var, timezone="Europe/Amsterdam"),
         utc=ibis.timestamp(_.var, timezone="UTC")
).mutate(
    es_ams = ibis.timestamp(_.var, timezone="Europe/Amsterdam").epoch_seconds(),
    es_utc = ibis.timestamp(_.var, timezone="UTC").epoch_seconds(),
    es_ams2 = _.var.cast("Timestamp('Europe/Amsterdam')").epoch_seconds(),
    es_utc2 = _.var.cast("Timestamp('UTC')").epoch_seconds(),
).limit(1)

plcon.execute(expr)
#           var                       ams                       utc      es_ams      es_utc     es_ams2     es_utc2
# 0  2023-01-02 2023-01-02 00:00:00+01:00 2023-01-02 00:00:00+00:00  1672614000  1672617600  1672614000  1672617600

duckcon.execute(expr)
#           var                       ams                       utc      es_ams      es_utc     es_ams2     es_utc2
# 0  2023-01-02 2023-01-02 01:00:00+01:00 2023-01-02 00:00:00+00:00  1672617600  1672617600  1672617600  1672617600

What version of ibis are you using?

10.4.0

What backend(s) are you using, if any?

Trino, DuckDB, polars

Relevant log output


Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

vspinu avatar May 14 '25 11:05 vspinu