bigcabbage

Return the same temp 'bqc' table when read a same BigQuery twice. BigQueryClient.class line 275 (TableInfo)destinationTableCache.get()

3

Hi, i tried to read a BigQuery table with query and do some transformation and then write back to the same table, and then do the process(read transform write) again...

Read Datetime type column in Dataframe drop trailing zeroes on second site

2

Data In BigQuery: id c_ts 2 2022-03-14T06:05:07 3 2022-03-14T00:00:00 1 2022-03-14T06:05:00 DDL CREATE TABLE `pypl-edw.pp_scratch.datetime_test` ( id STRING, c_ts DATETIME ) Code: val tableDataFrame = sparkSession.read.format(Constants.BIGQUERY).load("pypl-edw.pp_scratch.datetime_test") tableDataFrame.show() Dataframe: +---+-------------------+ |...