bigquery-emulator
bigquery-emulator copied to clipboard
Several issues when using with Node.js Bigquery client
Hey! Thank you for doing such an excellent job on this emulator! We are currently trying to migrate our test environment to the emulator. While it is partly blocked by another issue I left on your zetasql implementation, we're also experiencing some troubles with usage with Node.js client library.
- Inserting empty arrays to record fields and retrieving them after results in error.
Assume you have the following schema:
{
name: 'someArrayField',
mode: 'REPEATED',
type: 'STRING',
}
Then you try to insert an empty array using node client, querying it afterwards:
await dataset.table('table').insert([{ someArrayField: [] }])
await dataset.query(
`SELECT * FROM 'table'`,
);
This results in an error in the Node client lib as the emulator returns nothing for the array field:
bigquery_1 | 2023-01-23T13:15:10.609Z INFO contentdata/repository.go:138 {"query": "SELECT * FROM test1.example1", "values": []}
bigquery_1 | 2023-01-23T13:15:10.610Z DEBUG contentdata/repository.go:209 query result {"rows": [[[]]]}
return row.f.map((field, index) => {
^
TypeError: Cannot read properties of null (reading 'map')
at /Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/bigquery/build/src/bigquery.js:290:35
at Array.map (<anonymous>)
at mergeSchema (/Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/bigquery/build/src/bigquery.js:286:26)
at Array.map (<anonymous>)
at Function.mergeSchemaWithRows_ (/Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/bigquery/build/src/bigquery.js:283:14)
at /Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/bigquery/build/src/job.js:365:44
at /Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/common/build/src/util.js:412:25
at Util.handleResp (/Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/common/build/src/util.js:161:9)
at /Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/@google-cloud/common/build/src/util.js:534:22
at onResponse (/Users/vbshnsk/Desktop/extenda/hiiretail-txr-search-api/node_modules/retry-request/index.js:240:7)
- Sending number as a parameter value in query fails
Assume you have the following schema:
{
name: 'someNumberField',
mode: 'NULLABLE',
type: 'INT64',
}
Inserting data:
await dataset.table('table').insert([{ someNumberField: 123 }])
Selecting with param
await dataset.query(
`SELECT * FROM 'table' WHERE someNumberField = @someNumberField`,
{ params: { someNumberField: 123 } }
);
This results in a Bad Request error and following logs:
bigquery_1 | 2023-01-23T13:22:32.165Z ERROR server/handler.go:1007 invalid {"error": "invalid: json: cannot unmarshal number into Go struct field QueryParameterValue.Value of type string"}
- Cannot send array parameters
Assume you have the following schema:
{
name: 'someStringField',
mode: 'NULLABLE',
type: 'STRING',
}
Inserting data:
await dataset.table('table').insert([{ someStringField: "123" }])
Selecting with param
await dataset.query(
`SELECT * FROM 'table' WHERE someStringField = IN UNNEST(@someStringField)`,
{ params: { someStringField: ["123"]}, types: { someStringField: ['STRING']} }
);
Results in following log:
bigquery_1 | 2023-01-23T13:36:12.121Z INFO contentdata/repository.go:138 {"query": "SELECT * FROM test1.example1 WHERE someStringField = IN UNNEST(@someStringField)", "values": [{"Name":"someStringField","Value":""}]}
And error:
failed to parse statements: failed to parse statement: INVALID_ARGUMENT: Syntax error: Unexpected keyword IN [type.googleapis.com/zetasql.ErrorLocation='\x08\x01\x10\x36']
Node client sends this param with QueryParameter.arrayValues[]
: https://cloud.google.com/bigquery/docs/reference/rest/v2/QueryParameter#queryparametervalue
which is not handled in the emulator it seems
Also, it seems like actual BigQuery returns null on NULLABLE fields with no value, while the emulator does not return anything, which seems wrong.