Anoop Johnson

Results 11 comments of Anoop Johnson

> > Support for reading BigLake tables using BigQuery storage read API. > > Please remove the following dot. https://github.com/trinodb/trino/blob/master/.github/DEVELOPMENT.md#format-git-commit-messages > > Also, I would change to `Use BigQuery storage...

@ebyhr Do you have any more feedback or can this be merged?

@ebyhr @hashhar Friendly ping here. We have a GCP customer who is waiting for this PR to be merged.

> @anoopj what would the metadata look like for a sink export? Sink could be based on manifest files in [SymlinkTextInputFormat](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/SymlinkTextInputFormat.java). BigQuery also now [supports]( https://cloud.google.com/blog/products/data-analytics/bigquery-manifest-file-support-for-open-table-format-queries) manifest files. > I...

I am not aware of any existing tooling. BigQuery `LOAD` jobs support only file formats (Parquet/ORC/Avro et al) - open table formats are not supported.

@jaehyeon-kim That is correct. BigQuery Metastore is the replacement for BigLake Metastore. I recommend adjusting the roadmap to skip BigLake metastore and add support for BigQuery Metastore. This [PR](https://github.com/apache/iceberg/pull/11039) to...

@brokenjacobs We're working on a more feature-rich iteration of BigLake Metastore that is unified with BigQuery called BigQuery metastore. We expect this to be ready in the next few months...

> will it be a replacement of BigLake metastore and we should concentrate on this new BigQuery metastore and skip BigLake? Thank you for your contribution @vlad-lyutenko! Yes, BigQuery Metastore...

BigQuery Metastore follows the same API as BigQuery v2 REST API. From Java, it's just a `Table` with an `ExternalCatalogTableOptions`. [Javadocs](https://javadoc.io/doc/com.google.apis/google-api-services-bigquery/v2-rev20240323-2.0.0/com/google/api/services/bigquery/model/Table.html#getExternalCatalogTableOptions--) We will be updating [java-bigquery](https://github.com/googleapis/java-bigquery/tree/main/) client library also to...