spark-bigquery

spark-bigquery copied to clipboard

Hey, peeps! It would be great to enable authentication to BigQuery using an access token rather than supplying credentials. Looks like this is available in the beta system at [Google...

ntdesilv

Hi there, I am getting some differences when reading from BQ and writing to SQL Server. I tried locally, if i set .master("local") the read dataframe as 160k rows and...

celoibarros

Error with records that have a date of '01-01-0001'

2

ERROR: ``` com.miraisolutions.spark.bigquery.exception.IOException: BigQuery job JobId{project=gcp-gfs-datalake-process-tst, job=02f24e90-932f-4fec-8dcb-f86008e6a26a, location=US} failed with message: Error while reading data, error message: Invalid date value '-719164' for field 'vondt' of type 'DATE' ``` -719164 equates...

david-walker-gfs

spark empty dataframe with column not able to write in big query

1

Writing empty data frame with column name show following error ; Data frame as below +------+---------+-------+ |id|start|end| +------+---------+-------+ +------+---------+-------+ root |-- id: string (nullable = true) |-- start: long (nullable...

yogesh-0586