goccy
Connection refused is coming when connecting to Bigquery Emulator on Windows and Docker env on local
What happened?
Issue Coming on Windows and Docker local , But same code working on Mac when using big query emulator.
Library - "com.google.cloud:google-cloud-bigquery:2.42.3"
While writing the Data from CSV to bigquery using java code , It is giving error on windows and docker env. But it is working on mac. Bigquery Object - bigQuery = BigQueryOptions .newBuilder() .setHost(instanceAddress) .setProjectId(bigQueryConfigProperties.projectId) .setCredentials(NoCredentials.getInstance()) .build() .service Bigquery Writer- WriteChannelConfiguration.newBuilder(tableId) .setFormatOptions(FormatOptions.csv()) .setCreateSession(true) .build() writer.use { it.write(ByteBuffer.wrap(ClassPathResource(csvPathString).getInputStream().readBytes())) }
.BigQueryLocalDataLoader","timestamp":"2024-11-14T19:23:45.856Z"} com.google.cloud.bigquery.BigQueryException: Connection refused at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:116) at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.write(HttpBigQueryRpc.java:848) at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:59) at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:54) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:102) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:53) at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151)
What did you expect to happen?
It should load the csv data in bigquery on each OS type and between docker env.
How can we reproduce it (as minimally and precisely as possible)?
using library and docker image of Bigquery , Please use windows or docker env code that connects to bigquery emulator and inserts dat It will not allow to connect to bigquery emulator on local . But same code works over MAC machines.
Anything else we need to know?
I feel like there is some hard coding done for mac machines.
Hi Team, needed your assistance on above issue , It is a major blocker for us.
+1, did anyone find a solution or a workaround?
Edit: When specifying the host, I also included the port and it worked:
This does not work:
bq --api=http://localhost show --schema --format=prettyjson [project_id]:[dataset].[table]
This worked:
bq --api=http://localhost:9050 show --schema --format=prettyjson [project_id]:[dataset].[table]
Which in turn, I updated my options setup (I use it in the Test case), as:
options.setBigQueryEndpoint("http://localhost:9050");
Thanks for the reply, How ever it is not working , We need to use host.docker.internal as the host name instead of localhost. It is somewhere hard coded in the library. @XaviArnaus