bigquery-emulator icon indicating copy to clipboard operation
bigquery-emulator copied to clipboard

Serve HTTPS connections and enable JDBC connection

Open ohaibbq opened this issue 1 year ago • 8 comments

This PR adds an HTTPS endpoint and documentation on how to configure a JDBC connection to the emulator.

Closes #230 Closes #172

How to connect from JDBC

Data exploration tools such as JetBrains DataGrip or DBeaver are immensely valuable during development of projects using the emulator.

To connect the BigQuery Emulator to these tools from the JDBC, specify the following connection URL with your project ID:

jdbc:bigquery://https://localhost:9050;ProjectId=EMULATOR_PROJECT_ID;

The following options must also be specified:

Option Value Description
RootURL https://localhost:9070 This overwrites the URL used in the JDBC driver
SSLTrustStore /ssl/truststore.jks This must point to the JKS TrustStore that contains the SSL certificates for both the BigQuery Emulator, as well as oauth2.googleapis.com. A bundled truststore is located in the repository at ssl/truststore.jks. If using the Docker distribution, a volume can be created to access this data.
SSLTrustStorePwd test@123 This is the password for the trust store. The default password for the bundled store is test@123.

Known limitation: There is no documented method to bypass the JDBC OAuth flow, so a valid GCP OAuth credential is required to connect to the emulator. See Configuring Authentication in this document for a list of valid authentication methods. Exercise caution when using this feature with untrusted distributions of the BigQuery Emulator, as the OAuth Bearer token will be sent to the emulator server.

ohaibbq avatar Mar 05 '24 20:03 ohaibbq

I'd like to use https://go.dev/src/crypto/tls/generate_cert.go to generate certificates file ( no dependent openssl ). Also, please add the generation task to Makefile.

On the other hand, I don't know if it is necessary to include https support in the bigquery emulator itself. For example, it may be solved by preparing a reverse proxy that performs TSL termination.

goccy avatar Mar 14 '24 08:03 goccy

I had tried using mitmproxy and tinyproxy but couldn't figure out an easy solution.

I'll take a look at the crypto usage sometime soon.

ohaibbq avatar Mar 14 '24 17:03 ohaibbq

Hi @goccy, do we have plans to merge this PR? We are blocked on configuring a JDBC driver to connect to the emulator, and this PR will resolve the issue. Thanks!

hailie-l avatar Aug 16 '24 17:08 hailie-l