target-bigquery

target-bigquery copied to clipboard

I ran into issues with properties that had the following type ``` {'items': {'type': 'object'}, 'type': ['null', 'array']} ``` because BQ was expecting a schema for the objects, but `target-bigquery`...

rubenvereecken

When I run meltano with tap-intercom target-bigquery, it extracts and loads the data correctly to BQ. However, in the logs I see an error for every batch that says that...

blancaruano

When I run `meltano tap-coingecko target-jsonl` I am able to load raw json fields with `th.AnyType()` or `th.CustomType({"anyOf": [{"type": "object"}, {"type": "array"}, {}]})`. However when using `target-bigquery` (`target-snowflake` fails as...

melgazar9

Loader is unable to cope with JSON column from postgres

4

I am trying to use this target (via meltano, using most recent releases of both) to process data from postgres (https://github.com/transferwise/pipelinewise-target-postgres), specifically from a table that has two JSON columns,...

lukeasrodgers

gRPC fails to load data using `storage_write_api`

1

Specifically, using `TARGET_BIGQUERY_DEBUG=true`, I get the error: ``` google.api_core.exceptions.NotFound: 404 Requested entity was not found. Entity: projects/summitsift/datasets/test_data/tables/users/streams/_default ``` I don't know how gRPC works well enough to investigate this error....

rubenvereecken

`credentials_json` is not stored in the `.env` file When I use `meltano config target-bigquery set credentials_json [value]`, it isn't storing the credentials in the .env file. Instead, it is plain...

emiliemyth

Nested record fields failing to load with storage_write

2

I am experiencing an issue with loading records which have nested fields using `BigQueryStorageWriteDenormalizedSink` To replicate error: `cat outfile.jsonl | meltano invoke target-bigquery` Using outfile: ` {"type":"SCHEMA","stream":"invoices","schema":{"properties":{"Type":{"type":["null","string"]},"Contact":{"properties":{"ContactID":{"type":["string"]},"Name":{"type":["null","string"]}},"type":["null","object"],"additionalProperties":false},"InvoiceID":{"type":["string"]}},"type":["null","object"],"additionalProperties":false},"key_properties":["InvoiceID"]} {"type": "RECORD", "stream":...

alexcartwright

Schema evolution error when `denormalized` and `snake_case` is enabled

2

I'm running into issues to load data with the denormalised parameter enabled. Looking [into the code](https://github.com/z3z1ma/target-bigquery/blob/3952428bb043d1db80250e1f11cbcad9c8b18ecd/target_bigquery/core.py#L565C1-L569C14), it seems that even when the table does not yet exists in BigQuery, we...

qgab-flowdesk

Discussion: Parallelism in target-bigquery

1

## Parallelism in target-bigquery I'm opening this issue to discuss the current implementation of parallelism. ### Current Workflow The target currently supports both CPU parallelism and Thread parallelism. When a...

mattsax

Bumps [google-auth](https://github.com/googleapis/google-auth-library-python) from 2.17.3 to 2.21.0. Release notes Sourced from google-auth's releases. v2.21.0 2.21.0 (2023-06-26) Features Add framework for BYOID metrics headers (#1332) (1a8f50c) Bug Fixes Pypy unit test build...

dependabot[bot]