shmig icon indicating copy to clipboard operation
shmig copied to clipboard

Add Google BigQuery support

Open synchrone opened this issue 1 year ago • 3 comments

I know this is untested for now, and it would be relatively tricky to set up a test case. Sending this to test the waters on feasibility of merging upstream.

synchrone avatar Oct 25 '24 12:10 synchrone

Could we use this container

ghcr.io/goccy/bigquery-emulator:0.4.3

to write tests?

Based on the sub process bug report, I am currently updating the shmig_test repo.

But in any case, yes I am interested in merging this.

mbucc avatar Oct 25 '24 13:10 mbucc

Great link, thank you!

But it won't work for shmig until https://github.com/goccy/bigquery-emulator/issues/48 is resolved.

We could of course pretend there is information_schema in the test code :smile:

synchrone avatar Oct 25 '24 14:10 synchrone

more incompatibilities of bigquery-emulator after faking INFORMATION_SCHEMA.TABLES view with a manually created table:

+ bq_cli 'create table `shmig_version`(version int not null primary key not enforced, migrated_at timestamp default current_timestamp not null);'
+ local 'args=--format csv --dataset_id test --api http://127.0.0.1:9050 --project_id test query --nouse_legacy_sql'
+ [[ 1 -ne 0 ]]
+ /usr/bin/bq --format csv --dataset_id test --api http://127.0.0.1:9050 --project_id test query --nouse_legacy_sql

(ok we can get rid of default values and manually insert migrated_at)

+ bq_cli 'create table `shmig_version`(version int not null primary key not enforced, migrated_at timestamp not null);'
+ local 'args=--format csv --dataset_id test --api http://127.0.0.1:9050 --project_id test query --nouse_legacy_sql'
+ [[ 1 -ne 0 ]]
+ /usr/bin/bq --format csv --dataset_id test --api http://127.0.0.1:9050 --project_id test query --nouse_legacy_sql

it doesn't know what INT is?

At this point I consider the emulator unusable for this case. BigQuery would be absolutely free for running these test cases, and too easy to set up instead of fiddling around the emulator for a theoretical simplicity of not having to get an account and provide credentials for testing.

synchrone avatar Oct 31 '24 14:10 synchrone